AI Engineer with Deep Data Science Understanding

The project is set in the Scandinavian Banking industry and involves the development of AI solutions for a banking environment.

• Hands-on experience evaluating ML or LLM-based systems in production.
• Working knowledge of evaluation methodology, including error analysis and rubric design.
• Strong Python and data analysis skills.
• Solid experience with cloud-native ML infrastructure (e.g., AWS Bedrock).
• Ability to work independently in a developing team environment.

Nice to have:

• Experience with AWS AgentCore, Strands Agents, or similar agent runtimes.
• Experience in a regulated industry (financial services or healthcare) and familiarity with GDPR or equivalent data protection regimes.

• Build components for the AgentCore and/or Amazon Quick and run evaluation frameworks for skills and tools builds, covering error analysis, rubric design, and LLM-as-judge calibration.
• Ensure the reliability and accuracy of the AI components developed.
• Help operationalize the Evaluate and Iterate phase, ensuring evaluation practices are robust across agent builds.
• Define and track quality and cost-per-outcome metrics for production agents, using tools such as Arize AX.
• Run structured error analysis on agent outputs to identify failure patterns and implement fixes.

On-site or remote work options are available.