Data Engineer (Azure Databricks) in the EU

Andersen is hiring for a project enhancing a cloud-based data platform and delivering scalable data solutions for drug development. The customer is an international company operating in the life sciences sector, focusing on developing and commercializing specialized products and solutions supported by research, technology, and partnerships. The project aims to strengthen data products for scientific and business stakeholders within the drug development domain, enabling scalable data processing and reliable analytics.

• Commercial experience in Data Engineering for 4+ years
• Strong hands-on experience with Azure Databricks, including Spark and Delta Lake
• Experience building and maintaining multi-layer data pipelines following the Landing → Raw → Enriched → Curated architecture
• Experience with schema enforcement and schema-drift handling
• Ability to implement data quality checks at ingestion
• Experience orchestrating pipelines using Databricks Jobs / Workflows, including dependencies, retries, alerts, and failure handling
• Experience with both batch and API-based data ingestion
• Strong hands-on experience with dbt
• Ability to monitor job failures, data freshness, and overall pipeline health
• Experience delivering data in line with defined SLAs
• Experience implementing alerting, logging, and retry strategies
• Experience designing and maintaining access control models
• Understanding of audit trails, data lineage, and end-to-end traceability
• Experience with logging and evidence generation for audit and compliance
• Understanding of segregation of duties
• Experience defining and maintaining data contracts
• Ability to create and maintain clear pipeline documentation
• Experience working across controlled Dev, Test, and Prod environments
• Ability to ensure traceability between requirements and implemented data solutions
• English level: Upper-Intermediate and above

Nice to have:

• Experience with Databricks
• Experience with SAP S/4HANA, Microsoft 365, Azure, SharePoint, and Enterprise integrations
• Experience with CI/CD and DevOps practices for data including Git-based development and branching strategies
• Experience building and maintaining CI/CD pipelines for data workflows
• Experience promoting code and data solutions across Dev, Test, and Prod environments
• Ability to integrate automated testing into deployment pipelines
• Experience with MLOps and AI pipelines on Databricks
• Exposure to regulated domains such as GxP, clinical trials, and pharmacovigilance
• Strong SQL engineering skills, including development of modular and reusable dbt models, tests, and documentation at Analytics Engineer depth
• Experience with dimensional data modeling including star schemas, One Big Table (OBT) approaches, and curated data marts
• Familiarity with Tabular Editor and metadata-as-code concepts
• Experience with Power BI dataset design, performance optimization, and maintaining KPI and semantic consistency
• Experience in pharmaceutical or biotechnology industry

• Building and maintaining multi-layer pipelines (Landing → Raw → Enriched → Curated) on Azure Databricks (Spark, Delta Lake)
• Orchestrating pipelines via Databricks Jobs / Workflows including dependencies, retries, and alerting
• Implementing batch and API-based data ingestion from multiple source systems
• Applying schema enforcement and handling schema drift
• Running data quality checks at ingestion
• Working hands-on with dbt for pipeline observability and data governance
• Monitoring job failures, data freshness, and pipeline health against SLA-based delivery targets
• Setting up alerts, logging, and retry strategies for production pipelines
• Working within controlled Dev/Test/Prod environments including access control, audit trails, pipeline documentation, and traceability to requirements
• Contributing to CI/CD and MLOps/AI pipelines on Databricks as needed

Your personal data is protected according to GDPR regulations. By law, providing personal data required under the Labor Code is mandatory for recruitment; other data is voluntary. Data will be processed to conduct recruitment and may be retained for possible claims or future recruitments. Consent for data processing can be withdrawn anytime.