Data Annotation Project Manager

Grupa Allegro Sp. z o. o. is leveraging AI technology by enabling business, technology, and research teams to secure and maintain high-quality annotated datasets and evaluation pipelines. The company operates with a large-scale technology ecosystem including over 1000 microservices, an open-source data bus (Hermes), a Service Mesh, extensive data sets, and production-used machine learning.

• Demonstrated ability to manage data annotation projects with sustainable processes
• Practical knowledge of data annotation and quality assurance methods
• Good understanding of machine learning concepts, tasks, and metrics
• Understanding of applicable statistical concepts
• Strong verbal and written communication skills in Polish and English
• Experience managing annotation guidelines
• High attention to detail and commitment to data accuracy
• High organizational and problem-solving skills
• Proficiency in workflow software such as JIRA and Excel

• Manage data annotation projects across multiple teams, coordinating release schedules and consensus on issues
• Own end-to-end lifecycle of high-quality data acquisition, annotation, and curation
• Manage project budgets, design domain taxonomies, and establish evaluation benchmarks with ML engineers and product leaders
• Leverage GenAI tools, LLM-as-a-judge frameworks, and process automation to streamline labeling workflows
• Maintain data governance policies and drive scalable dataset curation
• Prepare and improve annotation guidelines based on feedback and quality metrics
• Implement and improve quality assurance processes for annotation pipelines
• Analyze annotated datasets for quality, biases, and risks
• Occasionally annotate data and provide feedback for tool improvement

Nice to have:

• Degree in Linguistics or STEM related fields
• Use metrics to evaluate research or business outcomes
• Basic knowledge of scripting languages, regular expressions, GitHub, and coding agents for automation