Senior Databricks Engineer - Governance

Brak informacji o wynagrodzeniu

SeniorHybryda·WarszawaFull-time·Umowa o pracę

#472550·Dodano 2 dni temu·0

Źródło: justjoin.it

Aplikuj teraz

Tech Stack / Keywords

DataAWS Azure Python SQL SparkUnity Catalog

Podsumowanie oferty

As a senior Databricks engineer, you will own and implement the governance layer of client platforms, working on pipelines, data models, platform setup, and governance controls. You will work with data and AI governance consultants to implement access, lineage, quality, and AI asset governance using Databricks, Unity Catalog, and MLflow.

Wymagania

5+ years of hands-on Databricks engineering in production, including at least two years designing and administering Unity Catalog at enterprise scale across several workspaces and teams.
Deep knowledge of Unity Catalog, including its privilege model, row and column security, tags and attribute-based policies, system tables, lineage, Delta Sharing, Lakehouse Federation, and migration from Hive metastore.
Strong Spark, SQL, and Python skills; experience with Delta Lake, Lakeflow pipelines and jobs, infrastructure as code with Terraform and Databricks Asset Bundles, and CI/CD.
Working knowledge of cloud security on Azure or AWS as it applies to Databricks, including identity, storage access, private networking, and key management.
Databricks Certified Data Engineer Professional or Associate certification.
Ability to explain an access model or lineage graph to a non-technical data owner; fluent English.

Nice to have:

Platform Administrator accreditation.
Interest in data governance and regulation; experience with a governance or catalog tool such as Microsoft Purview or Collibra, or with GDPR, the EU AI Act, or sector-regulated data.
Experience in financial services, energy, pharma, or the public sector.

Obowiązki

Design and implement Unity Catalog for clients, including metastore and catalog structure, workspace binding, external locations, storage credentials, identity federation, and ownership models.
Implement and document access controls, including privileges, groups, row filters, column masks, tag-based policies, and data classification; test controls so auditors can review them.
Produce platform governance evidence, including lineage, audit logs, billing information, access reviews, and dashboards; integrate with Microsoft Purview or Collibra where used by the client.
Implement data quality in pipelines using expectations in Lakeflow pipelines and Lakehouse Monitoring, and publish quality metrics to the catalog and governance team.
Govern AI assets with the AI governance team, including models and functions registered in Unity Catalog, approval workflows in MLflow, AI Gateway policies for LLM endpoints, and traceability logging.
Implement sovereignty and residency controls with the sovereignty team, including region placement, customer-managed keys, private connectivity, Delta Sharing, and Lakehouse Federation.
Migrate clients from Hive metastore and legacy permission models to Unity Catalog.
Build reusable assets, including Unity Catalog reference setups as code using Terraform and Databricks Asset Bundles, access policy patterns, and evidence dashboards; contribute to the governance-by-design reference architecture.

Benefity

Yearly financial bonus.
Private medical care with Medicover; additional packages, including dental, senior care, and oncology, are available on preferential terms.
Life insurance.
Access to the NAIS benefit platform.
Access to over 70 training tracks with certification opportunities on the NEXT platform.
Free access to the Education First languages platform, TED Talks, and Udemy Business materials and trainings.
Home office package including a laptop, monitor, and chair.

Premie

Opieka zdrowotna

Ubezpieczenie

Szkolenia wewnętrzne

Kursy językowe

Capgemini

780 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz