Senior Software Engineer - Akamai Inference Cloud

The Akamai Inference Cloud team develops AI platforms for inference models and applications within Akamai's Cloud Technology Group. The Inference Execution & Runtimes team manages the inference engine, AI runtime frameworks, model serving infrastructure, and execution environment, impacting latency, throughput, and efficiency of AI workloads across Akamai's global infrastructure.

• Relevant experience and a Bachelor's degree or equivalent in Computer Science or related field.
• Proficiency in Python and at least one systems programming language such as C++, Go, or Rust.
• Understanding of natural language processing concepts including tokenization, encoding, and text processing pipelines.
• Familiarity with AI inference, model serving, or LLM deployment including inference frameworks (TensorRT, vLLM, TorchServe, Triton).
• Experience building high-throughput, low-latency data processing systems or services.
• Familiarity with Linux systems, containerized environments, and profiling or debugging tools.
• Willingness to learn and grow within the AI inference and model serving field.

• Develop and maintain prompt processing and tokenization pipelines that prepare inference requests for efficient model execution.
• Implement request routing, scheduling, and batching logic to optimize throughput and latency across concurrent inference workloads.
• Contribute to the integration of new model architectures and serving backends into the runtime framework.
• Write well-tested, well-documented code and participate in code reviews to maintain high engineering standards across the inference stack.
• Support operational readiness through monitoring, debugging, and performance analysis of runtime components.