Azure Platform & Data Engineer

The role is within the Consumer Packaged Goods (CPG) industry, specifically in the dairy sector. It involves managing and enhancing the Azure infrastructure and platform capabilities combined with data engineering activities, focused on ensuring reliable and efficient data workflows in the Azure ecosystem.

• Strong experience with Terraform.
• Experience in Azure infrastructure management.
• Knowledge of Azure networking.
• Proficiency in Azure storage solutions.
• Understanding of Azure security and IAM.
• Experience with Azure DevOps, GitHub, and GitHub Actions.
• CI/CD experience.
• Proficiency in Pyspark and SQL.

Nice to have:

• Experience with Databricks administration.
• Familiarity with Databricks automation bundles.
• Experience leveraging Azure Data Factory.
• Knowledge of Data Lake architecture.

• Design, build, and maintain Azure-based platform services and infrastructure.
• Define and maintain Infrastructure-as-Code standards using Terraform.
• Manage Azure networking, storage, security, identity, and access management.
• Troubleshoot infrastructure and platform-related issues.
• Drive automation and standardization across development and operational environments.
• Manage and maintain Databricks workspaces and related services.
• Configure and govern clusters, access controls, and workspace standards in Databricks.
• Support platform performance, reliability, and operational stability.
• Establish best practices for Databricks usage and deployment.
• Develop and maintain CI/CD pipelines in Azure DevOps.
• Automate platform provisioning and deployment processes.
• Support release management and environment governance.
• Improve monitoring, logging, and operational tooling.
• Develop and maintain data ingestion and transformation pipelines.
• Work with Azure Data Factory, Databricks, and Data Lake technologies.
• Build and optimize ETL/ELT processes.
• Support data integration from internal and external source systems.