Data Engineer (Microsoft Fabric)

Core is a high-growth Microsoft Managed Services Provider (MSP) delivering technology solutions across Azure, Microsoft 365, Security, and Modern Workplace. They specialise in services, solutions, and software optimizing Microsoft investments for clients. Core focuses on enabling clients to unlock productivity, reduce risk, and transform their businesses through technology.

• Proven experience designing and delivering enterprise-grade data pipelines and lakehouse architectures.
• Strong understanding of medallion architecture and incremental/delta processing patterns.
• Hands-on experience with Microsoft Fabric components including Data Factory, Lakehouse, Dataflows Gen2, Notebooks, and Fabric capacity management.
• Hands-on experience with Databricks (notebooks, workflows, cluster configuration, and resource management).
• Experience with Azure SQL, Azure Storage Accounts, Azure Key Vault, and building orchestration pipelines.
• Working knowledge of Power BI including semantic models, DAX, and Power Query.
• Solid grasp of data pipeline best practices: modular and metadata-driven design, version control, and CI/CD.

Desirable:

• Exposure to Fabric IQ (Microsoft Fabric) and Genie (Databricks) for AI-driven data analysis.
• Experience with other low-code or analytics platforms.
• Knowledge of data governance frameworks and security models.
• Relevant certifications such as Fabric Data Engineer Associate (DP-700) and Azure Databricks Data Engineer Associate (DP-750).

• Lead design and architecture of end-to-end data platforms using Microsoft Fabric and/or Databricks.
• Translate business requirements into scalable data models and lakehouse architectures, following medallion (bronze/silver/gold) design principles.
• Design and implement incremental data processing patterns to maintain efficient and cost-effective pipelines.
• Build and optimize data pipelines using Data Factory, Lakehouse, Dataflows Gen2, and Notebooks in Fabric and workflows, Delta Live Tables, and notebooks in Databricks.
• Manage compute resources including sizing/scaling Databricks clusters and managing Fabric capacity.
• Collaborate with BI developers and business analysts to ensure data supports accurate reporting, including in Power BI.
• Support presales activities like solution design, effort estimation, and client presentations.
• Establish and promote best practices in data governance, performance tuning, and DevOps/CI-CD for data platforms.