Data Engineer (Azure Databricks)

16k - 24k PLN16 000 - 24 000 PLN/ mies./ mies.UoPUmowa o pracę (brutto)

SeniorZdalnie·WarszawaFull-time·Umowa o pracę

#472224·Dodano 3 dni temu·3

Źródło: justjoin.it

Aplikuj teraz

Tech Stack / Keywords

Databricks Azure Spark

Podsumowanie oferty

The role focuses on building and enhancing data products for scientific and business stakeholders in the drug development domain. You will develop a cloud-based data platform and scalable data solutions using Azure Databricks, Spark, and Delta Lake.

Wymagania

At least 4 years of commercial Data Engineering experience.
Strong hands-on experience with Azure Databricks, including Spark and Delta Lake.
Experience building and maintaining multi-layer data pipelines following the Landing → Raw → Enriched → Curated architecture.
Experience with schema enforcement and schema-drift handling.
Ability to implement data quality checks at ingestion.
Experience orchestrating pipelines using Databricks Jobs / Workflows, including dependencies, retries, alerts, and failure handling.
Experience with batch and API-based data ingestion.
Strong hands-on experience with dbt.
Ability to monitor job failures, data freshness, and overall pipeline health.
Experience delivering data in line with defined SLAs.
Experience implementing alerting, logging, and retry strategies.
Experience designing and maintaining access control models.
Understanding of audit trails, data lineage, and end-to-end traceability.
Experience with logging and evidence generation for audit and compliance purposes.
Understanding of segregation of duties.
Experience defining and maintaining data contracts.
Ability to create and maintain clear pipeline documentation.
Experience working across controlled Dev, Test, and Prod environments.
Ability to ensure traceability between requirements and implemented data solutions.
English at Upper-Intermediate level or above.

Nice to have:

Experience with Databricks.
Experience with SAP S/4HANA, Microsoft 365, Azure, SharePoint, and enterprise integrations.
Experience with CI/CD and DevOps practices for data, including Git-based development and branching strategies.
Experience building and maintaining CI/CD pipelines for data workflows.
Experience promoting code and data solutions across Dev, Test, and Prod environments.
Ability to integrate automated testing into deployment pipelines.
Experience with MLOps and AI pipelines on Databricks.
Exposure to regulated domains such as GxP, clinical trials, and pharmacovigilance.
Strong SQL engineering skills, including development of modular and reusable dbt models, tests, and documentation at Analytics Engineer depth.
Experience with dimensional data modeling, including star schemas, One Big Table (OBT) approaches, and curated data marts.
Familiarity with Tabular Editor and metadata-as-code concepts.
Experience with Power BI dataset design, performance optimization, and maintaining KPI and semantic consistency.
Experience in the pharmaceutical or biotechnology industry.

Obowiązki

Responsibilities:

Build and maintain multi-layer pipelines (Landing → Raw → Enriched → Curated) on Azure Databricks, using Spark and Delta Lake.
Orchestrate pipelines with Databricks Jobs / Workflows, including dependencies, retries, and alerting.
Implement batch and API-based data ingestion from multiple source systems.
Apply schema enforcement, handle schema drift, and run data quality checks at ingestion.
Work hands-on with dbt for pipeline observability and data governance tasks.
Monitor job failures, data freshness, and pipeline health against SLA-based delivery targets.
Set up alerts, logging, and retry strategies for production pipelines.
Work in controlled Dev/Test/Prod environments, including access control, audit trails, pipeline documentation, and traceability to requirements.
Contribute to CI/CD and MLOps/AI pipelines on Databricks where needed.

Benefity

Opportunity to earn up to an additional EUR 1,000 per month, depending on expertise, included in the annual bonus for participation in company activities.
Access to the corporate training portal and company knowledge base.
Certification compensation, including for AWS and PMP certifications.
Referral program.
Private health insurance and sports compensation, depending on employment type.
Corporate events and activities, including parties, pizza days, PlayStation, fruit, coffee, snacks, and movies.
Mentoring and adaptation systems for new employees.

Spotkania integracyjne

Dofinansowanie szkoleń

Opieka zdrowotna

Karta sportowa

Premie

Napoje w biurze

Darmowe przekąski

Andersen

30 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz