Principal Python + LangGraph Engineer

We are partnering with a US-based health-tech company on the takeover of a production AI-powered mobile coaching platform. The platform is built around a Python AI core (atlas-ai) which runs a FastAPI chat surface with intent routing and tool-using agents, a LangGraph-based agent framework with multi-agent orchestration, a Celery + Redis task queue, MongoDB for fitness-plan storage, Redis for conversation state, and Postgres for LangGraph checkpoints. Several agent personas support onboarding, chat, plan creation, workout adjustments, and habit formation with direct OpenAI integration for the LLM layer.

• 7+ years of professional experience with Python at senior+ level
• Deep experience with LangGraph including state graphs, checkpoints, interrupts, multi-agent supervision, and subgraphs
• Strong knowledge of the LangChain ecosystem: chains, tools, memory, output parsers, callbacks
• Production experience with FastAPI: streaming responses, dependency injection, middleware, async patterns
• Experience with Celery and Redis as task broker in production: task ordering, retries, idempotency, priority queues, dead-letter handling
• Concurrency in Python using asyncio (gather, structured concurrency, cancellation), threading boundaries, mixing sync and async code safely
• Multi-datastore operations combining MongoDB, Redis, and PostgreSQL with transaction boundaries
• Working with OpenAI API at scale including rate limits, retries with exponential backoff, fallback model routing, streaming, and tool/function calling
• Knowledge of agent design patterns such as ReAct, plan-and-execute, supervisor patterns, tool-use loops, multi-turn state, and interrupt resumption
• Disciplined prompt engineering including evaluation, A/B testing, version control, regression detection
• Token cost optimization techniques: prompt caching (Anthropic-style), model tiering, context window trimming, summary memory
• Production LLM observability: per-route token spend, prompt-level tracing, drift monitoring
• Testing discipline using pytest (including pytest-asyncio), property-based testing, snapshot tests for prompts, eval-based tests for agents
• Fluency with Pydantic v2 and type-hinted code throughout

Nice to have:

• Experience with Retrieval-Augmented Generation (RAG) and vector stores like Pinecone, Qdrant, pgvector
• Production incident command experience for LLM-powered systems
• Background in ML engineering (model serving, feature engineering)
• Experience with Anthropic / Claude API
• Data pipeline experience with Airflow, Dagster, Prefect
• Domain knowledge in fitness, health, or wearables
• Experience integrating Python core with mobile JSI / native bridges

• Audit atlas-ai: agent flows, LangGraph state machines, Celery topology, datastore usage, OpenAI integration patterns
• Produce a written assessment of operational risk: failure modes, race conditions, retry semantics, idempotency, checkpoint integrity
• Quantify token cost per agent flow and per user session
• Identify the highest-risk subsystems and propose stabilization plans
• Build or harden an evaluation harness for agent flows, including golden cases, regression suites, hallucination and safety tests
• Lead knowledge-transfer sessions from the client's AI team
• Set the technical direction for the AI core
• Lead design for new agent flows and major changes to existing flows
• Own the production health of the AI surface with platform and SRE support
• Hire and mentor the AI squad (~10 engineers at full scale)
• Represent the AI core in cross-team architecture conversations with the client