Senior Deep Learning Software Engineer, Inference

NVIDIA is a leading company in Artificial Intelligence, High-Performance Computing, and Visualization. The team develops and maintains high-performance deep learning frameworks such as SGLang and vLLM for efficient large-scale model serving and inference across NVIDIA accelerators.

• Master's or PhD degree or equivalent experience in Computer Engineering, Computer Science, EECS, or AI.
• Over 5 years of relevant software development experience.
• Excellent C/C++ programming and software design skills; Agile software development skills are helpful; experience with Python is a plus.
• Prior experience with training, deploying, or optimizing inference of deep learning models in production is a plus.
• Background in performance modeling, profiling, debugging, code optimization, or architectural knowledge of CPU and GPU is a plus.

Nice to have:

• Contributions to Deep Learning software projects such as PyTorch, vLLM, and SGLang.
• Experience with Multi-GPU communications technologies like NCCL and NVSHMEM.
• Experience building and shipping products to enterprise customers.
• GPU programming experience with CUDA, OAI TRITON, or CUTLASS.

• Performance optimization, analysis, and tuning of deep learning models in domains like LLM, Multimodal, and Generative AI.
• Scaling performance of deep learning models across various NVIDIA accelerator architectures.
• Contributing features and code to NVIDIA's inference libraries, including vLLM, SGLang, FlashInfer, and LLM software solutions.
• Collaborating with cross-collaborative teams across frameworks and NVIDIA libraries for inference optimization.