Craftware

Senior Data Engineer (Python, Databricks)

Brak informacji o wynagrodzeniu

SeniorZdalnie·WarszawaFull-time

#416012·Dodano dwa miesiące temu·13

Źródło: justjoin.it

Aplikuj teraz

Tech Stack / Keywords

Python PySpark Databricks SQLAzure DevOpsCI/CDDelta LakeYAML

Podsumowanie oferty

In this senior role, you will work on building and maintaining data engineering pipelines on Databricks using Python and PySpark. The position involves ensuring environment consistency, assessing new data sources for feasibility, and developing transformation logic following company standards. You will also define data reconciliation and acceptance criteria to support quality gates.

Wymagania

Senior hands-on experience with Azure Databricks & PySpark: jobs, clusters, Unity Catalog, transformation logic, unit testing
Experience with Delta Lake: schema evolution, MERGE, time travel; DAB: YAML config and bundle deployment
Advanced SQL skills; able to read 10-year-old stored procedures independently
Practiced CI/CD using Azure DevOps, Git branching strategy, and PR discipline
Ability to profile source data, define acceptance criteria, and manage data problem resolution from source assessment to UAT sign-off without a BA intermediary
Comfortable with incomplete documentation and evolving scope; document decisions clearly to avoid handover calls

Obowiązki

Build and maintain DAB pipelines and YAML-based transformation configuration; migrate toward native Databricks Asset Bundle patterns
Keep DEV / QAS / PRD environments consistent and drift-free across delivered markets
Assess new data sources before sprint commitment — shape, volume, quality risks — and write feasibility notes that gate scope entry
Develop PySpark transformations across Raw, Quality Integration, and Curated zones following CDA layer standards
Define and run data reconciliation and acceptance criteria so business users aren't the primary quality gate at UAT

Benefity

B2B contract
Fully remote service delivery
Broad range of projects including internal and international with a variety of clients and tasks
Budget for skills development and certifications
Regular collaboration reviews and project scope discussions
Additional benefits as part of collaboration
Networking and team-building events for project teams

Inne informacje

Zgodnie z art. 13 ust. 1 i 2 rozporządzenia Parlamentu Europejskiego i Rady (UE) 2016/679 z dnia 27 kwietnia 2016 r. w sprawie ochrony osób fizycznych w związku z przetwarzaniem danych osobowych i w sprawie swobodnego przepływu takich danych oraz uchylenia dyrektywy 95/46/WE (ogólne rozporządzenie o ochronie danych, dalej: RODO) ... Na działania Administratora przysługuje Panu/Pani skarga do Prezesa Urzędu Ochrony Danych Osobowych, ul. Stawki 2, 00-193 Warszawa.

Craftware

23 aktywne oferty

Zobacz wszystkie oferty

Aplikuj teraz