Senior Deep Learning Engineer, Accuracy Evaluation

NVIDIA is a fast-growing company at the forefront of the AI revolution, working with enterprise-grade GPU clusters capable of hundreds of PetaFLOPS to deliver advanced AI models and directly impacting the AI landscape and NVIDIA's roadmap.

• BS, MS, or PhD in Computer Science, Machine Learning, Statistics, or related field.
• 6+ years hands-on experience with LLMs and designing/running evaluations for large language or multimodal AI systems including agentic, multi-turn, or reasoning-heavy settings.
• Strong statistical foundations including experimental design, significance testing, and regression analysis.
• Proven experience building evaluation infrastructure such as pipelines, benchmark harnesses, and reproducible CI systems.
• Clear communication skills to translate quantitative evaluation results into decisions for researchers, product teams, and leadership.

Nice to have:

• Deep familiarity with open-source evaluation frameworks.
• Experience designing evaluations for agentic systems including tool use, multi-turn reasoning, and interactive benchmarks.
• Publication or contribution record in evaluation research, benchmark design, or reproducibility analyses.
• Experience with low-precision inference (FP8, INT4, quantization-aware settings) and understanding calibration and sparsity effects.
• Comfort running large-scale workloads on HPC/Slurm clusters, including reproducible experiment management with tools like MLflow and W&B.

• Design and build decision-grade evaluation environments for NVIDIA's frontier models producing auditable accuracy signals for major releases.
• Research and develop new evaluation methodologies for emerging model families and capability domains.
• Build and operate evaluation infrastructure and pipelines including benchmark environments, regression CI systems, and statistical analysis tools.
• Collaborate with model research, training, and customer teams to translate evaluation signals into release and training decisions.

NVIDIA is an equal opportunity employer; it does not discriminate based on race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability, or other protected characteristics by law.