Junior Data Engineer (Databricks)

Addepto is a leading AI consulting and data engineering company that builds scalable, ROI-focused AI solutions for large enterprises and startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. The company focuses exclusively on Artificial Intelligence and Big Data, helping organizations unlock the potential of their data through systems designed for measurable business impact and long-term growth. Addepto also develops its own product offerings and releases open-source solutions to the community. It has been recognized by Forbes as one of the top 10 AI consulting companies worldwide.

• At least 1 year of proven commercial experience developing or maintaining Big Data systems.
• Hands-on experience with Big Data technologies including Databricks, Apache Spark, Airflow, and DBT.
• Strong programming skills in Python, including writing clean code and OOP design.
• Experience in designing and implementing data governance and data management processes.
• Experience implementing and deploying solutions in cloud environments, preferably Azure.
• Practical knowledge of DevOps practices, including designing and maintaining CI/CD pipelines for data and ML workflows, and Terraform for Infrastructure as Code.
• Knowledge of building and deploying Power BI reports and dashboards for data visualization.
• Excellent understanding of dimensional data and data modeling techniques.
• Excellent communication skills and consulting experience with direct client interaction.
• Ability to work independently and take ownership of project deliverables.
• Bachelor’s or Master’s degree in Computer Science, Data Science, Mathematics, Physics, or related field.

Nice to have:

• Experience with CI/CD, Kubernetes, Kafka, Power BI, Dagster, dbt.

• Design scalable data processing pipelines for streaming and batch processing using Big Data technologies like Databricks, Airflow, and/or Dagster.
• Contribute to the development of CI/CD and MLOps processes.
• Develop applications to aggregate, process, and analyze data from diverse sources.
• Collaborate with the Data Science team on Machine Learning projects, including text/image analysis and predictive model building.
• Develop and organize data transformations using Databricks/DBT and Apache Airflow.
• Translate business requirements into technical solutions and ensure optimal performance and quality.

Projects:

• Design and development of a universal data platform for global aerospace companies using Azure and Databricks.
• Data Platform Transformation for an energy management association, implementing a data catalog and secure integration solutions with Databricks and Apache Spark/PySpark.
• Design of data transformation and data ops pipelines for a global car manufacturer, handling real-time streaming and batch data, including data cleaning, normalization, and optimization for performance and accuracy.