Azure Data Engineer with Databricks

LinkGroup offers roles focused on Azure and Databricks data engineering and data architecture.

• Hands-on experience implementing solutions with data lakes and pipelines using Databricks, Azure Data Factory.
• Proven experience designing and implementing solutions in Azure cloud; knowledge of AWS and GCP clouds is an advantage.
• Proficiency in SQL, PySpark, data modeling (3NF, star/snowflake schemas), and ELT/ETL design patterns; familiarity with ETL tools including Azure Data Factory.
• Experience with Python or similar scripting languages for data transformation, orchestration, and automation.
• Familiarity with advanced Databricks features (Unity Catalog, MLflow, Jobs & Pipelines).
• Understanding of cloud data architecture principles including data Lakehouse concepts, Medallion architecture, API integration, and event-driven architecture.
• Experience with CI/CD and DevOps practices for data workflows using tools such as Git, Terraform, Azure DevOps.
• Excellent communication and client-facing skills with ability to explain technical concepts clearly.
• Strong collaboration ability across multidisciplinary teams.
• Comprehensive knowledge of Azure Databricks and Microsoft Fabric architecture, including core components, integration strategies, and best practices for scalable, secure, and efficient data solutions.
• Ability to translate business needs into technical solutions and work collaboratively across teams.

Data Engineer Responsibilities:

• Develop and implement scalable data processing solutions on Azure, transforming and cleaning large structured and unstructured datasets efficiently.
• Design, build, and maintain robust data pipelines integrating data from diverse sources (databases, APIs, streaming platforms).
• Collaborate with cross-functional teams including data engineers, data scientists, and cloud architects to deliver end-to-end data platform solutions.
• Engage at all critical stages of project development with business analysts, source system experts, data architects, engineers, and product owners.
• Clearly communicate and document technical work, factoring in performance and cost considerations in all design decisions.

Data Architect Responsibilities:

• Design and implement scalable cloud-based data warehousing and analytics solutions using Azure Data Lake, Azure Databricks, Microsoft Fabric, and related services (e.g., Power BI).
• Define best practices for data governance, security, performance optimization, and cost efficiency in Azure and Databricks environments.
• Collaborate with cross-functional teams (data engineers, data scientists, and cloud architects) to deliver end-to-end data platform solutions.
• Integrate Databricks with the broader Azure ecosystem, including data lakes, AI/ML frameworks, reporting systems, and data catalogs (e.g., Unity Catalog, Purview).
• Advise clients on modernization strategies, including migration from legacy on-premises databases to Azure and Databricks-based architectures.
• Support pre-sales and solutioning activities, including estimation, proof-of-concepts, and technology assessments.
• Stay up to date with innovations in Azure, Databricks, and Fabric and apply industry trends to ongoing projects.
• Collaborate across all crucial stages of project development with business analysts, source system experts, data architects, engineers, and product owners.
• Propose and defend effective, consistent solutions balancing strategic development and existing architecture.
• Communicate and document work clearly, considering performance and cost factors in design decisions.