AI Evaluation Engineer (Python, QA or Security)

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.

• 5+ years in software development.
• Experience with Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
• Experience writing functional and integration tests.
• English proficiency at B2 level or higher.

• Build realistic developer environments with codebase, infrastructure, and contextual information representing a virtual company.
• Design evaluation tasks from intermediate states of these environments, crafting prompts and defining solvability criteria.
• Write tests that verify agent solutions, accepting all valid approaches and rejecting incorrect ones.
• Iterate on tasks and tests based on QA feedback by reviewing solutions, analyzing failures, and refining evaluations.

Please submit your CV in English and indicate your level of English proficiency.