Senior Data Engineer (Spark)

Addepto is a leading AI consulting and data engineering company focused exclusively on Artificial Intelligence and Big Data. They build scalable, ROI-focused AI solutions for major enterprises and innovative startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. The company develops its own product, ContextClue, and contributes open-source solutions to the AI community. Addepto is recognized by Forbes as a top 10 AI consulting company worldwide and is part of KMS Technology, a US-based global technology group.

• At least 5 years of commercial experience implementing, developing, or maintaining Big Data systems, data governance, and data management processes.
• Strong programming skills in Python (or Java/Scala) with clean code and OOP design.
• Hands-on experience with Big Data technologies: Spark, Cloudera, Airflow, Iceberg, Kafka, and CI/CD.
• Excellent understanding of dimensional data and data modeling techniques.
• Experience deploying solutions in cloud environments.
• Consulting experience with strong communication and client management skills.
• Ability to work independently and take ownership of project deliverables.
• Fluent English (at least C1 level).
• Bachelor’s degree in technical or mathematical studies.

Nice to have:

• Experience with Kubernetes, DevOps, Docker, Databricks, NiFi.

• Develop and maintain a high-performance data processing platform for automotive data, ensuring scalability and reliability.
• Design and implement data pipelines for processing large volumes of data in streaming and batch modes.
• Optimize data workflows using technologies such as Spark, Cloudera, and Airflow.
• Manage structured and unstructured data efficiently with data lake technologies like Iceberg.
• Collaborate with cross-functional teams to ensure seamless integration of data sources.
• Monitor and troubleshoot the platform to maintain high availability, performance, and accuracy.
• Leverage cloud services (AWS) for infrastructure management and workload scaling.
• Write and maintain high-quality Python (or Java/Scala) code for data processing and automation.

Tools used include: Jira, Confluence, Wiki, GitHub, Agile methodologies such as Scrum and Kanban.