AI Evaluation Engineer (Testing oraz Development)

Brak informacji o wynagrodzeniu

MidHybryda·KrakówFull-time·B2B·Umowa o pracę

#477193·Dodano 5 dni temu·1

Źródło: justjoin.it

Aplikuj teraz

Tech Stack / Keywords

Python Machine LearningRAGAI

Podsumowanie oferty

The role focuses on testing, evaluating, and improving AI agents and Generative AI applications, while developing supporting services and integrations. You will build evaluation frameworks and monitoring tools, including Python-based services, and validate RAG systems.

Wymagania

Strong experience with Python development.
Hands-on experience testing AI, Machine Learning, or Generative AI systems.
Understanding of Large Language Models (LLMs), AI agents, and RAG architectures.
Experience creating test scenarios, evaluation datasets, and automated validation frameworks.
Knowledge of API testing and automation tools.
Experience working with cloud environments such as Azure or Google Cloud Platform.
Strong analytical mindset and ability to investigate complex AI behaviors.

Obowiązki

Design, build, and maintain evaluation frameworks for AI agents and Generative AI solutions.
Define testing methodologies, benchmarks, and quality metrics for agent performance.
Evaluate AI outputs for accuracy, relevance, consistency, safety, and adherence to business requirements.
Identify and analyze hallucinations, failure cases, and context-related issues.
Create automated testing pipelines and regression testing strategies for GenAI applications.
Validate Retrieval Augmented Generation (RAG) systems, memory stores, and context-engineering mechanisms.
Collaborate with AI Engineers, Data Engineers, and Product teams to improve agent behavior and user experience.
Develop Python-based services and tools supporting evaluation and monitoring processes.
Analyze prompts and agent orchestration workflows to optimize performance and quality.
Support production monitoring and continuous improvement of AI solutions.

Benefity

Stable and long-term cooperation.
Well-defined career path at a European engineering and IT consulting company.
Opportunity to relocate and work in different ALTEN Polska branches.
Opportunity to engage in another project within the company after completing a project.
Introduction and cooperation with a dedicated Competences Manager.
Medicover medical care and dental care.
Medicover Benefits platform.
Medicover Sport card.
Referral Program.
Layette for a newborn employee’s child.
Group life insurance.
Pension scheme.
Access to professional certifications with technology partners, such as Google and ISTQB.
Access to relevant industry events.
Access to the Knowledge Sharing Program.

Opieka zdrowotna

Karta sportowa

Ubezpieczenie

ALTEN Polska

113 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz