Data Engineer (Spark)

15.1k - 21k PLN15 120 - 21 000 PLN/ mies./ mies.B2BB2B (netto)

MidZdalnie·WarszawaFull-time·B2B

#465445·Dodano 5 dni temu·0

Źródło: nofluffjobs.com

Aplikuj teraz

Tech Stack / Keywords

Python SQL Spark CI/CD Kubernetes KafkaNiFiDocker DatabricksMLOpsDevOpsIcebergdbt

Podsumowanie oferty

As a Data Engineer, you will develop and maintain data platforms and pipelines for projects in automotive, aerospace and telecommunications. The work includes processing data in streaming and batch modes and building reporting platforms, using technologies including Spark, Python, Airflow, Iceberg and cloud platforms.

Wymagania

At least 3 years of commercial experience implementing, developing or maintaining Big Data systems, data governance and data management processes.
Strong programming skills in Python (or Java/Scala), including clean code and OOP design.
Hands-on experience with Big Data technologies such as Spark, Cloudera, Data Platform, Airflow, Kafka, NiFi, Docker and Iceberg.
Excellent understanding of dimensional data and data modeling techniques.
Experience implementing and deploying solutions in cloud environments.
Consulting experience, excellent communication and client management skills, including direct client interaction as a consultant.
Ability to work independently and take ownership of project deliverables.
Fluent English (at least C1 level).
Bachelor’s degree in technical or mathematical studies.

Nice to have:

Experience with an MLOps framework such as Kubeflow or MLflow.
Familiarity with Databricks and/or dbt.

Obowiązki

Develop and maintain a high-performance data processing platform for automotive data, ensuring scalability and reliability.
Design and implement data pipelines that process large volumes of data in streaming and batch modes.
Optimize data workflows for data ingestion, processing and storage using Spark, Cloudera and Airflow.
Work with Iceberg to manage structured and unstructured data.
Collaborate with cross-functional teams to understand data requirements and integrate data sources.
Monitor and troubleshoot the platform to ensure availability, performance and data processing accuracy.
Use AWS services to manage infrastructure and scale processing workloads.
Write and maintain data-processing and automation code in Python (or Java/Scala).

Benefity

Language classes.
Sponsored training and conferences, including access to training materials and certifications through partnerships with Databricks and Anthropic.
Team-building events and an integration budget.
Medical and sports packages, eye care, and well-being support services, including psychotherapy and coaching.
Work equipment, including a laptop and other necessary devices.
A dedicated onboarding buddy.
Work-anniversary, birthday and milestone celebrations.
Referral bonus.
Paid leave.
Free coffee.

Kursy językowe

Dofinansowanie szkoleń

Budżet konferencyjny

Spotkania integracyjne

Opieka zdrowotna

Karta sportowa

Ubezpieczenie

Premie

Płatny urlop

Napoje w biurze

Szkolenia wewnętrzne

Addepto

62 aktywne oferty

Zobacz wszystkie oferty

Aplikuj teraz