Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.

• 8+ years in software development
• Core stack experience with Python (FastAPI), JavaScript, TypeScript (React), Docker, Postgres, Kafka, Redis
• Experience writing functional and integration tests
• English proficiency at B2 level or higher

• Build realistic developer environments including codebase, infrastructure, and context such as tickets, docs, and conversations
• Design evaluation tasks at intermediate states of these environments with clear prompts and solvability criteria
• Write tests that verify AI agent solutions, accepting all valid approaches and rejecting incorrect ones
• Iterate and refine tasks and tests based on quality assurance feedback by reviewing agent solutions and analyzing failures

Participation is project-based, not permanent employment. Tasks are estimated to take approximately 30 hours each with flexible scheduling. Compensation is up to $150/hr equivalent depending on level and pace. Candidates must submit CV in English and indicate their English proficiency level.