Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.

• 8+ years of experience in software development.
• Proficiency with Python (including FastAPI), JavaScript, TypeScript (including React), Docker, Postgres, Kafka, and Redis.
• Experience writing functional and integration tests.
• English proficiency at B2 level or higher.

• Build realistic developer environments including codebase, infrastructure, and context such as tickets, documentation, and conversations.
• Design tasks from intermediate states of these environments, craft prompts, and define solvability criteria.
• Write tests that verify AI agent solutions, accepting all valid approaches and rejecting incorrect ones without being too strict or lenient.
• Iterate on tasks and tests based on QA feedback through reviewing agent solutions, analyzing failures, and refining evaluations until fair and robust.

Participation is project-based, not permanent employment. Please submit your CV in English and indicate your level of English proficiency.