Senior Software Engineer - AI Coding Benchmarks

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.

• Minimum 5 years of experience in software development.
• Core technology stack experience including Python (with FastAPI), JavaScript/TypeScript (with React), Docker, Postgres, Kafka, and Redis.
• Experience writing functional and integration tests.
• English language proficiency at B2 level or higher.

• Build realistic developer environments including codebase, infrastructure, and context such as tickets, documentation, and conversations.
• Design tasks from intermediate states of these environments, crafting prompts, defining completion criteria, and ensuring tasks are solvable by AI agents.
• Write tests to verify AI agent solutions, ensuring all valid approaches are accepted and incorrect solutions rejected.
• Iterate on tasks and tests based on quality assurance feedback by reviewing agent solutions, analyzing failures, and refining evaluations for fairness and robustness.

Please submit your CV in English and indicate your level of English proficiency.