AI focused DataBricks Developer (Digital Engineering)

Our client is a leading life insurance company, part of a global financial services group. The company offers a wide range of products, including annuities and life insurance, delivered through a large nationwide network of agents.

• 3+ years of experience as a Data Engineer
• Strong experience with Databricks ecosystem (Architecture, Unity Catalog, Delta tables, lineage, permissions, Databricks Genie, notebooks)
• Strong experience with Python
• Solid proficiency in SQL
• Experience working in Azure cloud environment
• Experience with Databricks pipelines AI automation (Mosaic AI or AWS Agentcore)

Nice to have:

• Experience with Azure data integration tools (Synapse Pipelines, Azure Data Factory)
• Experience with Azure SQL pools
• Understanding of performance tuning for data workloads
• Experience with CI/CD, DevOps practices, Infrastructure as Code (IaC)

• Develop and maintain data pipelines within the Azure-based data platform
• Work with the Databricks ecosystem (Delta Tables, Unity Catalog, notebooks) to process and manage data
• Build and optimize data transformations using Python and SQL
• Develop and support AI-driven data pipelines and automation using Databricks (Mosaic AI or similar tools)
• Support and enhance existing production data systems
• Collaborate with a team of engineers on Databricks-based solutions
• Ensure data quality, consistency, and reliability across data pipelines