Senior Data Engineer / Tech Lead (Spark)

Addepto is a leading AI consulting and data engineering company that builds scalable, ROI-focused AI solutions for large enterprises and pioneering startups. The company has developed its own product - ContextClue - and contributes open-source solutions to the AI community. As part of KMS Technology, Addepto combines deep AI specialization with enterprise-scale delivery capabilities to support clients in moving from AI experimentation to production impact.

• At least 4 years of commercial experience in Data Engineering or Big Data projects
• Strong programming skills in Python
• Very good knowledge of SQL
• Strong hands-on experience with Apache Spark and distributed data processing
• Practical experience with workflow orchestration tools, preferably Apache Airflow
• Experience with Apache Iceberg and/or modern lakehouse architectures
• Experience working with Docker and Kubernetes
• Good understanding of data modelling, data pipelines, and data processing architectures
• Experience working with relational databases and/or query engines such as PostgreSQL or Trino
• General understanding of cloud environments and cloud-based data solutions
• Ability to take ownership of technical solutions and contribute to architectural and technical decisions
• Experience supporting other engineers through technical guidance, knowledge sharing, or mentoring
• Ability to work independently and take ownership of project deliverables
• Strong communication and collaboration skills
• Fluent English (at least C1 level)
• Bachelor’s degree in technical or mathematical studies

Nice to have:

• Experience with CI/CD, Kafka, NiFi, Java, Scala, Databricks, DevOps, Cloudera

• Design, develop, and maintain scalable data pipelines using Python, SQL, Spark, and Airflow
• Build and improve data processing solutions based on Apache Iceberg and modern lakehouse architectures
• Work with PyArrow, PyIceberg, PostgreSQL, and Trino
• Take technical ownership of selected areas and contribute to technical and architectural decisions
• Combine hands-on development with technical leadership and knowledge sharing within the team
• Build and maintain containerized workloads using Docker and Kubernetes
• Ensure data pipelines are scalable, reliable, maintainable, and production-ready
• Collaborate with engineers and other stakeholders to translate business and technical requirements into effective solutions
• Contribute to engineering best practices, including code quality, testing, CI/CD, and automation
• Monitor, troubleshoot, and continuously improve existing data pipelines and platform components
• Use modern development practices, including AI-assisted development, to improve engineering efficiency