AI Tester / AI Evaluation Engineer

We are part of Hiberus, a technology company founded in Spain with a strong presence in the Polish market: 4,700+ specialists, operating in 14+ countries and delivering projects for clients worldwide. We work with technologies including AI, Data, Cloud, Software Development, BI, and Cybersecurity. One of our key advantages is Hiberus University — an international learning environment offering real opportunities to develop and expand your skills. More than 1,000 people were trained last year in selected technologies. Your first project is just the beginning. With Hiberus, you can build a long-term career.

• Minimum 3 years of experience in software testing, analytical systems or data-driven solutions
• Practical knowledge of creating test cases, test scenarios and defect reports
• Basic understanding of ML or LLM models and the specifics of testing non-deterministic systems
• Ability to design test suites for evaluating model output quality, classification and information extraction
• Knowledge of SQL, Postman and API testing tools
• Ability to evaluate model outputs based on accuracy, completeness, source consistency, security and usability

Nice to have:

• Experience testing RAG systems, chatbots, AI assistants or predictive models
• Knowledge of automated LLM evaluation tools and prompt regression testing
• Experience in GenAI security testing, including prompt injection and jailbreak testing
• Knowledge of accessibility, UX or plain-language requirements
• English proficiency sufficient for analysing technical documentation

• Preparing test scenarios for AI, ML and GenAI solutions
• Designing question sets, edge cases and reference data for model evaluation
• Testing the quality of model outputs and the stability of AI system behaviour
• Performing regression testing after changes to the model, prompts, source data or configuration
• Reporting defects, anomalies and quality-related risks
• Collaborating with AI Engineers, analysts, business and security teams during solution acceptance
• Documenting test results and preparing recommendations regarding the quality of AI systems