Addepto

Senior Data Engineer (Spark)

21k - 28.6k PLN21 000 - 28 560 PLN/ mies./ mies.B2BB2B (netto)

SeniorZdalnie·WarszawaFull-time·B2B

#465985·Dodano 12 dni temu·0

Źródło: justjoin.it

Aplikuj teraz

Tech Stack / Keywords

IcebergSpark HadoopClouderaSQL PythonData IntegrationDatabricksDBTTerraform

Podsumowanie oferty

You will develop and maintain a high-performance data processing platform for automotive data, handling streaming and batch workloads. The role focuses on data pipelines and processing with Spark, Python, and related data technologies.

Wymagania

At least 5 years of commercial experience implementing, developing, or maintaining Big Data systems and data governance and management processes.
Strong programming skills in Python or Java/Scala, including clean code and OOP design.
Hands-on experience with Big Data technologies such as Spark, Cloudera, Data Platform, Airflow, Kafka, NiFi, Docker, and Iceberg.
Excellent understanding of dimensional data and data modeling techniques.
Experience implementing and deploying solutions in cloud environments.
Consulting experience, including direct client interaction, and strong communication and client management skills.
Ability to work independently and take ownership of project deliverables.
Fluent in English (at least C1 level).
Bachelor’s degree in technical or mathematical studies.

Nice to have:

Experience with an MLOps framework such as Kubeflow or MLflow.
Familiarity with Databricks and/or dbt.

Obowiązki

Develop and maintain a high-performance data processing platform for automotive data, ensuring scalability and reliability.
Design and implement data pipelines that process large volumes of data in streaming and batch modes.
Optimize data workflows for data ingestion, processing, and storage using Spark, Cloudera, and Airflow.
Work with data lake technologies such as Iceberg to manage structured and unstructured data.
Collaborate with cross-functional teams to understand data requirements and integrate data sources.
Monitor and troubleshoot the platform to ensure availability, performance, and data-processing accuracy.
Use AWS cloud services to manage infrastructure and scale processing workloads.
Write and maintain data-processing and automation code in Python or Java/Scala.

Benefity

Language classes.
Sponsored training and conferences, including training materials and certifications through partnerships with Databricks and Anthropic.
Team-building events and an integration budget.
Medical and sports packages, eye care, and well-being support services, including psychotherapy and coaching.
Full work equipment, including a laptop and other necessary devices.

Kursy językowe

Dofinansowanie szkoleń

Budżet konferencyjny

Spotkania integracyjne

Opieka zdrowotna

Karta sportowa

Addepto

56 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz