Data Engineer (Databricks)

Addepto is a leading AI consulting and data engineering company specializing exclusively in Artificial Intelligence and Big Data. The company develops scalable AI solutions for large enterprises and startups and has products like ContextClue. It is part of KMS Technology, a US-based global technology group, combining AI specialization with enterprise delivery capabilities.

• At least 3 years of commercial experience implementing or maintaining Big Data systems.
• Strong programming skills in Python with clean code and OOP design.
• Proficient in SQL, including performance tuning and experience with data warehousing.
• Experience designing and implementing data governance and management processes.
• Deep expertise with Databricks, Spark, Apache Airflow, and other data orchestration tools.
• Experience deploying solutions in cloud environments, preferably Azure.
• Knowledge of building and deploying Power BI reports and dashboards.
• Excellent understanding of dimensional data and data modeling techniques.
• Consulting experience with guidance on architecture, technology selection, and best practices.
• Ability to work independently and take ownership of deliverables.
• Master’s or Ph.D. in Computer Science, Data Science, Mathematics, Physics, or related field.

Nice to have:

• Experience with CI/CD, Kubernetes, Kafka, Airflow, Dagster, and dbt.

• Design and optimize scalable data processing pipelines for streaming and batch workloads using Databricks, Apache Airflow, and Dagster.
• Architect and implement end-to-end data platforms ensuring availability, performance, and reliability.
• Lead development of CI/CD and MLOps processes for deployment and model lifecycle management.
• Develop and maintain applications for aggregating and analyzing diverse data sources efficiently.
• Collaborate with Data Science teams on Machine Learning projects including text/image analysis and predictive models.
• Design and manage data transformations using Databricks, DBT, and Apache Airflow ensuring data integrity.
• Translate business requirements into scalable technical solutions ensuring data quality.
• Ensure data security, compliance, and governance best practices across pipelines.