AI Evaluation Engineer (Python, QA or Security)

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.

• 5+ years in software development.
• Experience with Python (specifically FastAPI), JavaScript/TypeScript (specifically React), Docker, Postgres, Kafka, Redis.
• Experience writing functional and integration tests.
• English proficiency at B2 level or higher.

• Build realistic developer environments including virtual company, codebase, infrastructure, and context such as tickets, docs, and conversations.
• Design tasks from intermediate states of environments, craft prompts, define what "solved" means, and ensure tasks are solvable by AI agents.
• Write tests to verify agent solutions, accepting all valid approaches and rejecting incorrect ones appropriately.
• Iterate on tasks and tests based on QA feedback by reviewing agent solutions, analyzing failures, and refining evaluations to ensure fairness and robustness.

Please submit your CV in English and indicate your level of English proficiency. Participation is project-based, not permanent employment.