Capgemini

Senior Databricks Engineer - Governance

Brak informacji o wynagrodzeniu

SeniorHybryda·WarszawaFull-time

#472687·Dodano 12 dni temu·1

Źródło: nofluffjobs.com

Aplikuj teraz

Tech Stack / Keywords

Databricks UNITYSecurityHive Spark SQL PythonInfrastructure as CodeTerraform CI/CDCloud securityAzure AWSStorageNetworkingGDPRAI

Podsumowanie oferty

You will own the governance layer of client platforms, working on Databricks engagements across pipelines, data models and platform setup. You will implement governance in Unity Catalog, including access models, lineage and audit evidence, in collaboration with data and AI governance consultants.

Wymagania

At least 5 years of hands-on Databricks engineering in production, including at least 2 years designing and administering Unity Catalog at enterprise scale.
Deep knowledge of Unity Catalog, including its privilege model, row and column security, tags and attribute-based policies, system tables, lineage, Delta Sharing, Lakehouse Federation, and migration from Hive metastore.
Strong Spark, SQL and Python skills; experience with Delta Lake, Lakeflow pipelines and jobs, infrastructure as code with Terraform and Databricks Asset Bundles, and CI/CD.
Working knowledge of cloud security on Azure or AWS as it applies to Databricks, including identity, storage access, private networking and key management.
Databricks Certified Data Engineer Professional or Associate certification.
Ability to explain an access model or lineage graph to a non-technical data owner; fluent English.

Nice to have:

Platform Administrator accreditation.
Experience with a governance or catalog tool such as Purview or Collibra, or with GDPR, the EU AI Act or sector-regulated data.
Experience in financial services, energy, pharma or the public sector.
Interest in data governance and regulation.

Obowiązki

Design and implement Unity Catalog for clients, including metastore and catalog structure, workspace binding, external locations and storage credentials, identity federation, and ownership models.
Build, test and document access controls that reflect governance designs, including privileges, groups, row filters, column masks, attribute-based access control and data classification.
Produce lineage, audit and billing evidence from system tables, and create access reviews and dashboards for data owners and compliance teams; integrate with enterprise catalogs where used.
Implement data quality checks in pipelines, including expectations in Lakeflow pipelines, Lakehouse Monitoring, and quality metrics for catalogs and governance teams.
Govern AI assets on the platform with the AI governance team, including models and functions registered in Unity Catalog, MLflow approval workflows, AI Gateway policies for LLM endpoints, and traceability logging.
Implement sovereignty and residency controls with the sovereignty team, including region placement, customer-managed keys, private connectivity, Delta Sharing and Lakehouse Federation.
Migrate clients from Hive metastore and legacy permission models to Unity Catalog, and address governance visibility gaps.
Build reusable Unity Catalog reference setups as code using Terraform and Databricks Asset Bundles, create access policy patterns and evidence dashboards, and contribute to governance-by-design reference architecture.

Benefity

Yearly financial bonus.
Private medical care with Medicover, with additional packages available on preferential terms.
Life insurance and access to the NAIS benefit platform.
Access to over 70 training tracks with certification opportunities on the NEXT platform.
Free access to the Education First languages platform, TED Talks and Udemy Business materials and trainings.
Home office package including a laptop, monitor and chair.

Premie

Opieka zdrowotna

Ubezpieczenie

Dofinansowanie szkoleń

Karta sportowa

Napoje w biurze

Parking dla rowerów

Parking dla aut

Telefon

Szkolenia wewnętrzne

Capgemini

792 aktywne oferty

Zobacz wszystkie oferty

Aplikuj teraz