Data Engineer (Databricks)

Addepto is a leading AI consulting and data engineering company focusing exclusively on Artificial Intelligence and Big Data. It serves major enterprises and startups globally and develops products like ContextClue. As part of KMS Technology, Addepto combines AI specialization with enterprise-scale capabilities.

• At least 3 years of commercial experience implementing, developing, or maintaining Big Data systems.
• Strong programming skills in Python, including clean code and OOP design.
• Proficient in SQL including performance tuning and query optimization.
• Experience with data warehousing and dimensional data modeling techniques.
• Expertise in Databricks, Spark, and modern data orchestration tools like Apache Airflow.
• Ability to implement and deploy solutions in cloud environments, preferably Azure.
• Knowledge of building and deploying Power BI reports and dashboards.
• Consulting experience guiding architectural decisions and technology selection.
• Ability to work independently and take ownership of deliverables.
• Master’s or Ph.D. in Computer Science, Data Science, Mathematics, Physics, or related field.

Nice to have:

• Experience with CI/CD, Kubernetes, Kafka, Airflow, Dagster, and dbt.

• Design and optimize scalable data processing pipelines for both streaming and batch workloads using Databricks, Apache Airflow, and Dagster.
• Architect and implement end-to-end data platforms ensuring high availability, performance, and reliability.
• Lead the development of CI/CD and MLOps processes to automate deployments and model lifecycle management.
• Develop and maintain applications for aggregating, processing, and analyzing data from diverse sources.
• Collaborate with Data Science teams on Machine Learning projects including feature engineering and predictive model deployment.
• Design and manage complex data transformations using Databricks, dbt, and Apache Airflow ensuring data integrity.
• Translate business requirements into scalable technical solutions ensuring optimal performance and data quality.
• Ensure data security, compliance, and governance best practices across all data pipelines.