Addepto

Lead Data Engineer (Spark)

21k - 31.9k PLN21 000 - 31 920 PLN/ mies./ mies.B2BB2B (netto)

SeniorZdalnie·WarszawaFull-time·B2B

#460815·Dodano 19 dni temu·4

Źródło: justjoin.it

Aplikuj teraz

Tech Stack / Keywords

AWS AirflowIcebergDocker Kubernetes Java SQL Python Big Data Apache Spark

Podsumowanie oferty

As a Lead Data Engineer, you will design and develop scalable data platforms for automotive and aerospace projects, including vehicle data processing and enterprise data platforms. The work covers streaming and batch processing with Apache Spark, alongside architecture, data governance, and data transformations.

Wymagania

5+ years of commercial experience implementing, developing, or maintaining Big Data systems.
Strong programming skills in Python or Java/Scala, including clean code and OOP design.
Experience designing and implementing data governance and data management processes.
Familiarity with Big Data technologies such as Spark, Cloudera, Airflow, NiFi, Docker, Kubernetes, Iceberg, Trino, or Hudi.
Proven expertise implementing and deploying solutions in cloud environments, preferably AWS.
Excellent understanding of dimensional data and data modeling techniques.
Excellent communication skills and consulting experience with direct client interaction.
Ability to work independently and take ownership of project deliverables.
Master’s or Ph.D. in Computer Science, Data Science, Mathematics, Physics, or a related field.
Fluent English (C1 level).

Obowiązki

Your main responsibilities:

Design and develop scalable data management architectures, infrastructure, and platform solutions for streaming and batch processing using Big Data technologies such as Apache Spark, Hadoop, and Iceberg.
Design and implement data management and data governance processes and best practices.
Contribute to the development of CI/CD and MLOps processes.
Develop applications to aggregate, process, and analyze data from diverse sources.
Collaborate with the Data Science team on data analysis and Machine Learning projects, including text and image analysis and predictive model building.
Develop and organize data transformations using dbt and Apache Airflow.
Translate business requirements into technical solutions and ensure optimal performance and quality.

Benefity

Language classes.
Sponsored training and conferences, including access to Databricks training materials and certifications.
20 fully paid days off.
Team-building events and an integration budget.
Medical and sports packages, eye care, and well-being support services, including psychotherapy and coaching.
Work equipment, including a laptop and other necessary devices.

Kursy językowe

Dofinansowanie szkoleń

Budżet konferencyjny

Płatny urlop

Spotkania integracyjne

Opieka zdrowotna

Karta sportowa

Addepto

56 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz