Azure Databricks Data Engineer

Capgemini is a global leader in partnering with companies to transform and manage their business by harnessing the power of technology. The Group has over 360,000 team members globally in more than 50 countries and offers expertise in cloud, data, AI, connectivity, software, digital engineering, and platforms.

• Hands-on experience designing and developing data pipelines with Databricks.
• Strong working knowledge of Delta Lake, Spark, and Databricks notebooks.
• Proficiency in SQL and Python for data transformation and analysis.
• Familiarity with data modeling, data warehousing, and data architecture principles.
• Understanding of data governance and security frameworks (experience with Unity Catalog is a plus).
• Experience with tools such as MLflow, Power BI, or similar technologies.
• Solid problem-solving and troubleshooting skills for data and pipeline issues.
• Effective communicator and collaborator in cross-functional environments.
• Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field.
• Databricks certification (e.g., Databricks Certified Data Engineer Associate or Professional) is an advantage.

Nice to have:

• Security knowledge
• UNITY
• MLflow

• Design, develop, and maintain ETL/ELT pipelines and workflows using Databricks.
• Collaborate with cross-functional teams to understand data requirements and deliver high-quality datasets.
• Build and manage data lake and data warehouse architectures to support analytics and reporting.
• Optimize data workflows for performance, scalability, and cost efficiency.
• Ensure data quality, consistency, and reliability across all data platforms.
• Apply data governance and security standards, leveraging tools such as Unity Catalog.
• Integrate Databricks with enterprise systems and tools (e.g., Delta Lake, MLflow, Power BI, Mosaik SDK, Agent Bricks).
• Monitor, test, and troubleshoot data pipelines to ensure production stability and performance.
• Contribute to best practices in data engineering and participate in code reviews.