Technical Lead, Senior AI Engineer - VonHalsky (m/f/n)

InPost Group is an innovative European out of home deliveries company, operating intelligent lockers across several countries. It is one of the largest out of home delivery providers in Europe, publicly traded with a market capitalization of about $5 billion as of March 2023, and employs over 10,000 people worldwide. The team focuses on building the best FMCG e-commerce platform in Poland, managing over 300,000 products and collaborating with more than 300 sellers, with a mobile application used by almost 1.5 million users in Poland.

Required:

• 5+ years building and running production software, including 2+ years on LLM-based systems with real user traffic.
• Strong engineering fundamentals with experience in testing, CI/CD, containers, observability, and cloud environments.
• Experience in evaluating generative systems with knowledge of LLM-as-judge, human-label calibration, regression suites, and eval methodologies.
• AI engineering fundamentals including prompt and context engineering, agent primitives in production, and eval/LLM-observability tooling such as LangFuse, Braintrust, Weave, or equivalents.
• Comfort with data at scale: SQL, data lakes or warehouses, and reproducible metric creation.
• Fluency with AI-assisted development tools like Claude Code, Cursor, and Copilot.
• Ability to communicate technical concepts to non-technical audiences effectively.
• Proficiency in English (B2) and Polish.

Nice to have:

• Experience in harness and loop engineering including agent loops, retries, fallbacks, and tool-call orchestration.
• Knowledge in auto-improving systems that close the loop from production signals back into product development.
• Experience with adversarial robustness, red-teaming, abuse and fraud detection.
• Domain experience in e-commerce, search, or recommendation systems.

• Own the LLM-judge pipeline for release gating using real and golden conversations.
• Manage evaluation datasets, including Polish-language coverage beyond generic benchmarks.
• Perform root-cause analysis on real conversations with a stable issue taxonomy.
• Develop and maintain the "sus" detector for abusive, adversarial, and anomalous sessions.
• Apply evals-driven development ensuring evaluation precedes feature implementation.
• Interface with product and business stakeholders to convert vague requests into measurable quality definitions.
• Set and defend the technical direction of the platform.
• Review code and mentor other engineers.
• Scope work with Project Manager and Engineering Manager.
• Present results to stakeholders and hold the line against ad-hoc requests affecting platform work.

Success criteria:

• Judge pass rate calibrated against human labels and trusted.
• Stable, versioned issue taxonomy live with comparable trends.
• Every release gated by reproducible eval runs.
• Golden set with documented coverage and identified blind spots.
• Platform operated and extendable by at least two other engineers besides you.