Senior AI Engineer (m/f/n)

InPost Group is an innovative European out of home deliveries company with operations across several countries. The company operates a network of intelligent lockers providing customers with fast, convenient, and secure delivery options. InPost Group has over 10,000 people worldwide and is one of the largest out of home delivery providers in Europe. The team builds the FMCG e-commerce platform in Poland, offering over 300,000 products, working with over 300 sellers, and a mobile app downloaded by almost 1.5 million users in Poland.

Required:

• At least 5 years of experience building and running production software with 2+ years on LLM-based systems.
• Strong engineering fundamentals including testing, CI/CD, containerization, observability, and cloud work, primarily using Python.
• Experience evaluating generative systems with expertise in LLM-as-judge, human-label calibration, inter-annotator agreement, and regression suites.
• Expertise in AI engineering fundamentals such as prompt and context engineering, agentic primitives, and eval/LLM-observability tooling (e.g., LangFuse, Braintrust, Weave).
• Comfort with data at scale including SQL, working with lakes or warehouses, and building reproducible metrics.
• Fluency with AI-assisted development tools like Claude Code, Cursor, and Copilot.
• Ability to communicate technical arguments to non-technical audiences.
• Proficiency in English (B2) and Polish due to interaction with user conversations.

Nice to have:

• Experience in harness and loop engineering around models including agent loops, retries, fallbacks, and tool-call orchestration.
• Knowledge of auto-improving systems that close production feedback loops.
• Experience with adversarial robustness, jailbreak testing, red-teaming, abuse, and fraud detection.
• Background in e-commerce, search, or recommendation domains.

• Own and improve the LLM-judge pipeline for release gating based on conversation evaluation.
• Develop and maintain eval datasets including real coverage across intents and Polish-language benchmarks.
• Perform root-cause analysis on real conversations to provide diagnostic insights with a stable issue taxonomy.
• Develop and manage the "sus" detector for abusive, adversarial, and anomalous sessions.
• Drive evals-driven development, writing evaluations before feature implementation.
• Serve as the interface to product and business teams, translating vague requests into measurable quality definitions.

Leadership responsibilities:

• Set and defend technical direction for the Evaluations Platform.
• Act as reviewer of record for the platform area.
• Mentor other engineers.
• Scope work collaboratively with Product Manager and Engineering Manager.
• Present results to stakeholders and manage platform priorities against ad-hoc requests.