AI Engineer (Senior)

This role involves working on a long-term project for the banking sector at one of the largest financial institutions in Poland, focusing on agent-based projects using LLM in a regulated environment.

• Experience designing prompts for LLM in a production environment
• Understanding of agent architectures and patterns such as ReAct, planning, reflection, and self-check
• Ability to design modular prompts for reusability and versioning
• Experience with tool use / function calling and system integrations
• Knowledge of RAG, grounding, and working with long context
• Ability to define and analyze quality metrics of responses and agent trajectories
• Experience in LLM evaluation (golden datasets, LLM-as-a-judge, regression testing)
• Knowledge of typical LLM risks (prompt injection, jailbreaks, PII, data leakage) and mitigation techniques
• Ability to analyze agent traces and conduct root cause analysis
• Experience working in regulated or high-security environments
• Good collaboration skills with business and technical teams
• Fluent communication in English in a project environment

• Participate in a long-term banking sector project at a major financial institution in Poland
• Work on agent-based projects using LLM (PoC → pilot → UAT) in a regulated environment
• Design prompts treated as production code with versioning, review, and CI/CD
• Create agent architectures using tool use / function calling and multi-step decision flows
• Work with RAG based on banking regulations, procedures, and documents
• Design outputs validatable for downstream systems (JSON, decision schemas, tables)
• Build and maintain golden datasets and evaluation sets for prompts and agent trajectories
• Participate in regression quality tests, UAT, and analysis of quality and cost metrics
• Work in Azure OpenAI (enterprise) environment and experiment with reasoning models
• Collaborate with product owners, analysts, AI/data/IT teams, and compliance, risk, and security areas
• Work in a hybrid model: 2 days per week in the Warsaw office

Work in a hybrid model: 2 days per week in the Warsaw office