Generative AI Operations Engineer (GenAI Ops)

• 3+ years in a DevOps, SRE, or MLOps role focused on cloud infrastructure with experience in cloud services (AWS, GCP, Azure)
• Skills in building and managing CI/CD pipelines (Jenkins, GitLab CI, or cloud-native services) and proficiency in scripting languages (e.g., Python, Bash)
• Familiarity with IaC tools (AWS CDK, CloudFormation, Terraform) and containerization/orchestration (Docker, Kubernetes)
• Experience deploying and operating LLM inference frameworks (vLLM, Triton, TGI, Ray Serve, KServe/Seldon)
• Hands-on experience with LLM/app tracing and metrics tools (OpenTelemetry + Langfuse, Arize Phoenix, WhyLabs) and building evaluation pipelines
• Experience operating retrieval pipelines: embedding generation, indexing/refresh strategies, vector databases (Pinecone, Weaviate, Milvus, FAISS), and relevance monitoring
• Experience in managing multi-agent workflows (LangGraph, CrewAI, AutoGen-like), including state management, retries, rate limiting, tool-failure handling, and auditing
• Implementing guardrails such as secrets isolation, tool/API permissions, prompt-injection defenses, data leakage prevention, PII redaction, and policy enforcement
• Background integrating agents with external tools using MCP or similar tool-calling standards is a plus

Nice to have:

• Master's degree or PhD in Computer Science, AI, Machine Learning, or related fields
• Experience with cloud-native GenAI services like AWS Bedrock, Azure AI Foundry, or Google Vertex AI
• Familiarity with LLM architecture and operational challenges
• Experience designing/managing multi-agent or orchestrated workflows
• Knowledge of monitoring/observability tools (Prometheus, Grafana, Datadog)
• Relevant cloud or DevOps certifications
• Strong problem-solving skills in fast-paced, collaborative environments

• Build and manage robust, automated CI/CD pipelines for training, evaluating, and deploying large language models (LLMs) and AI agents
• Design, deploy, and manage sophisticated multi-agent systems enabling Agent-to-Agent (A2A) communication and collaboration to automate complex business processes
• Implement and manage secure, scalable integrations between AI agents and external tools/APIs utilizing open standards such as the Model Context Protocol (MCP)
• Utilize AI-powered development tools to accelerate infrastructure code writing, testing, and troubleshooting in cloud environments
• Use Infrastructure as Code (IaC) services or tools like Terraform to define and manage GenAI workload infrastructure
• Implement model monitoring and logging to track performance, resource utilization, and system health, including tracing agent actions and multi-step conversational flows
• Design scalable architectures for model serving and inference with continuous optimization for performance and cost-effectiveness
• Implement and enforce security best practices and ensure compliance with industry standards and regulations for GenAI infrastructure and data

EPAM is a global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 and startups. With over 30 years in software, product, and platform engineering, EPAM empowers clients to become AI-Native enterprises, driving innovation and digital investment value.