Staff Software Engineer, Inference

CoreWeave is The Essential Cloud for AI™, providing a platform of technology, tools, and teams enabling innovators to build and scale AI with confidence. The company powers massive-scale GPU workloads for AI/ML, VFX, rendering, and real-time inference with a Kubernetes-native inference platform. Founded in 2017 and publicly traded since March 2025, CoreWeave focuses on infrastructure as a core product operated at massive scale.

• Bachelor’s degree in Computer Science, Engineering, or related technical field (or equivalent practical experience).
• 8+ years of experience building large-scale distributed systems or cloud platforms.
• Proven track record of leading cross-team or organisation-level technical initiatives at scale.
• Strong coding proficiency in Go, Python, or C++.
• Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design.
• Strong understanding of networked systems, performance optimisation, and distributed system design.
• Hands-on experience with inference systems including batching/micro-batching strategies, caching, memory optimisation, mixed precision (BF16/FP8), and streaming token delivery.
• Experience improving tail latency (P95/P99) and platform reliability through metrics-driven engineering.
• Proven experience owning system-wide SLIs/SLOs, capacity planning, and autoscaling strategies.

• Operate as a technical leader across multiple teams and services in CoreWeave's Kubernetes-native inference platform.
• Drive architecture, performance, and reliability initiatives for request routing, adaptive scheduling, GPU resource management, and cost-per-token optimisation under strict P99 SLAs.
• Implement advanced inference optimisations such as speculative decoding and KV-cache reuse.
• Establish performance benchmarking frameworks.
• Guide cross-functional alignment across infrastructure boundaries.
• Raise engineering rigour and observability practices across the organisation.
• Mentor senior and mid-level engineers.

Employment offers are conditional upon satisfactory results from a basic criminal record check conducted in compliance with GDPR. This position requires access to export controlled information; applicants must be either U.S. persons as defined or eligible to access without authorization, or reasonably likely to obtain required export authorization. CoreWeave is an equal opportunity employer committed to inclusive hiring. Unsolicited CVs sent by recruitment agencies will be considered a "free gift" with no fee obligations.