Data Engineer

Core is a Microsoft Managed Services Provider (MSP) specializing in Azure, Microsoft 365, Security, and Modern Workplace solutions. The company delivers modern technology solutions to optimize clients' Microsoft investments, focusing on productivity, risk reduction, and business transformation.

• Proven experience designing and delivering enterprise-grade data pipelines and lakehouse architectures.
• Strong understanding of medallion architecture and incremental/delta processing patterns.
• Hands-on experience with Microsoft Fabric (Data Factory, Lakehouse, Dataflows Gen2, Notebooks) including capacity management.
• Hands-on experience with Databricks (notebooks, workflows, cluster configuration, and resource management).
• Experience with Azure SQL, Azure Storage Accounts, Azure Key Vault, and building orchestration pipelines.
• Working knowledge of Power BI (semantic models, DAX, Power Query) supporting reporting.
• Knowledge of data pipeline best practices: modular and metadata-driven design, version control, and CI/CD.

Nice to have:

• Exposure to Fabric IQ (Microsoft Fabric) and Genie (Databricks) for AI-driven data analysis.
• Experience with other low-code or analytics platforms.
• Knowledge of data governance frameworks and security models.
• Relevant certifications such as Fabric Data Engineer Associate (DP-700) and Azure Databricks Data Engineer Associate (DP-750).

• Lead the design and architecture of end-to-end data platforms using Microsoft Fabric and/or Databricks.
• Translate business requirements into scalable data models and lakehouse architectures following medallion design principles.
• Design and implement incremental data processing patterns for efficient pipelines.
• Build and optimize data pipelines using Fabric components (Data Factory, Lakehouse, Dataflows Gen2, Notebooks) and/or Databricks (Workflows, Delta Live Tables, Notebooks).
• Manage compute resources by sizing and scaling Databricks clusters and Fabric capacity.
• Collaborate with BI developers and business analysts to ensure curated data supports accurate reporting in Power BI.
• Support presales activities including solution design, effort estimation, and client presentations.
• Establish and promote best practices in data governance, performance tuning, and DevOps/CI-CD for data platforms.

The controller of your personal data is Core Technology Systems (U.K.) Limited, located in London. Data is processed for recruitment purposes based on relevant legal grounds including Labour Code and GDPR. Providing mandatory data is required, other data is voluntary. Data retention limits and rights to access, rectification, deletion, restriction, objection, and complaints are described. Consent to future recruitment can be withdrawn without affecting prior processing. Data is not subject to profiling.