Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.

• 8+ years in software development.
• Core stack includes Python (specifically FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
• Experience writing functional and integration tests.
• English proficiency at B2 level or higher.

• Build realistic developer environments including codebase, infrastructure, and context such as tickets, documentation, and conversations.
• Design tasks from intermediate states of these environments by crafting prompts and defining solvability for AI agents.
• Write tests that verify AI agent solutions, accepting all valid approaches and rejecting incorrect ones.
• Iterate tasks and tests based on feedback from quality assurance, reviewing AI agent solutions, analyzing failures, and refining evaluations.

Please submit your CV in English and indicate your level of English proficiency.