Senior Software Engineer, AI Inference Systems

NVIDIA is advancing AI research and development to create technologies that enable anyone to harness the power of AI. The team consists of experts in AI, systems, and performance optimization, led by world-renowned experts in AI systems with multiple academic and industry research awards.

• Bachelor’s degree (or equivalent experience) in CS, CE, or SE with 7+ years of experience; or Master’s degree with 5+ years; or PhD with thesis and top-tier publications in ML Systems, GPU architecture, or high-performance computing.
• Strong programming skills in Python and C/C++; experience with Go or Rust is a plus.
• Solid CS fundamentals: algorithms & data structures, operating systems, computer architecture, parallel programming, distributed systems, deep learning theories.
• Knowledge and passion for performance engineering in ML frameworks (e.g., PyTorch) and inference engines (e.g., vLLM and SGLang).
• Familiarity with GPU programming and performance: CUDA, memory hierarchy, streams, NCCL; proficiency with profiling/debug tools (e.g., Nsight Systems/Compute).
• Experience with containers and orchestration (Docker, Kubernetes, Slurm); familiarity with Linux namespaces and cgroups.
• Excellent debugging, problem-solving, and communication skills.

• Contribute features to vLLM to empower the newest models with the latest NVIDIA GPU hardware features.
• Profile and optimize the inference framework (vLLM) using methods like speculative decoding, data/tensor/expert/pipeline-parallelism, prefill-decode disaggregation.
• Develop, optimize, and benchmark GPU kernels using techniques such as fusion, autotuning, and memory/layout optimization.
• Build and extend high-level DSLs and compiler infrastructure to boost kernel developer productivity.
• Define and build inference benchmarking methodologies and tools; contribute to the MLPerf Inference benchmarking suite.
• Architect scheduling and orchestration of containerized large-scale inference deployments on GPU clusters across clouds.
• Conduct and publish original research to advance ML Systems; integrate research ideas and prototypes into NVIDIA’s software products.