Senior Data Engineer

Brak informacji o wynagrodzeniu

SeniorZdalnie·KrakówFull-time

#471119·Dodano 5 dni temu·1

Źródło: Softeta

Aplikuj teraz

Tech Stack / Keywords

Airflow SparkdbtBigQuery PythonGCP KMSTinkGoogle Cloud Data CatalogApache KafkaAEAD

Podsumowanie oferty

As a Senior Data Engineer, you will lead the architecture, migration, and optimization of production data pipelines for a telecommunications project. The pipelines use Airflow, Spark, dbt, BigQuery, and Python.

Wymagania

Deep hands-on experience with Airflow, Spark, dbt, BigQuery, and Python.
Proven track record leading a live pipeline migration technically from conception to completion.
Hands-on experience refactoring production pipelines while verifying identical output.
Demonstrated success establishing engineering best practices, including testing, code reviews, and incident resolution, within a team.
Concrete mentoring or technical leadership experience.

Nice to have:

Previous experience with encryption at rest, data retention policies, or regulatory compliance (e.g., GDPR).
Envelope encryption using key management services (GCP KMS), BigQuery AEAD functions, or Tink.
BigQuery column-level security, policy tags, and Google Cloud Data Catalog.
Event-driven architectures using Apache Kafka.

Obowiązki

Propose and lead the architecture for user-data workflows across Airflow, Spark, dbt, and BigQuery, including encryption points, derived tables, backfills, and partitions.
Sequentially migrate existing pipelines behind a custom regression harness to guarantee identical output.
Establish engineering standards after migration, including code reviews, testing protocols, incident post-mortems, and release checklists.
Implement verification jobs to validate end-to-end data lifecycle rules.
Participate in pipeline maintenance, incident response, and root-cause resolution.
Document design choices using Architecture Decision Records (ADRs).

Benefity

Training budget for professional and personal growth.
Flexible working hours.
Technical equipment that you can choose.

Dofinansowanie szkoleń

Elastyczne godziny

Softeta

6 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz