Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.

• 8+ years of experience in software development.
• Proficiency in core technologies including Python (FastAPI), JavaScript, TypeScript (React), Docker, Postgres, Kafka, and Redis.
• Experience writing functional and integration tests.
• English proficiency at B2 level or higher.

• Build realistic developer environments including a virtual company with codebase, infrastructure, and context such as tickets, documentation, and conversations.
• Design tasks from intermediate states of environments by crafting prompts, defining completion criteria, and ensuring solvability by AI agents.
• Write functional and integration tests that verify solutions from AI agents, accepting all valid approaches and rejecting incorrect ones.
• Iterate on tasks and tests based on QA feedback, reviewing agent solutions, analyzing failures, and refining evaluations to ensure fairness and robustness.

Please submit your CV in English and indicate your level of English proficiency.