Senior Data Engineer (Spark)
Tech Stack / Keywords
Firma i stanowisko
Addepto is a leading AI consulting and data engineering company that builds scalable, ROI-focused AI solutions for large enterprises and startups. It specializes exclusively in Artificial Intelligence and Big Data, working with clients such as Rolls Royce, Continental, Porsche, ABB, and WGU. The company is part of KMS Technology, a global technology group. Addepto also develops its own AI product, ContextClue, and contributes to open-source AI solutions.
Wymagania
- At least 3 years of commercial experience implementing, developing, or maintaining Big Data systems, data governance, and data management.
- Strong programming skills in Python, or Java/Scala, with clean code and OOP design.
- Hands-on experience with Big Data technologies such as Spark, Cloudera, Airflow, Kafka, NiFi, Docker, and Iceberg.
- Excellent understanding of dimensional data and data modeling techniques.
- Experience deploying solutions in cloud environments.
- Consulting experience with strong communication and client management skills.
- Ability to work independently and take ownership of project deliverables.
- Fluent in English (minimum C1 level).
- Bachelor’s degree in technical or mathematical studies.
Nice to have:
- Experience with MLOps frameworks like Kubeflow or MLflow.
- Familiarity with Databricks and dbt.
Obowiązki
- Develop and maintain a high-performance data processing platform for automotive data, ensuring scalability and reliability.
- Design and implement data pipelines that process large volumes of data in both streaming and batch modes.
- Optimize data workflows for efficient ingestion, processing, and storage using Spark, Cloudera, and Airflow.
- Work with data lake technologies such as Iceberg to manage structured and unstructured data.
- Collaborate with cross-functional teams to understand data requirements and integrate data sources.
- Monitor and troubleshoot the platform to ensure high availability, performance, and data accuracy.
- Leverage cloud services (AWS) for infrastructure management and workload scaling.
- Write and maintain high-quality code in Python, Java, or Scala for data processing and automation.
Benefity
- Work in a supportive team passionate about AI & Big Data.
- Engage with global enterprises and startups on international projects.
- Flexible work arrangements with remote and office options.
- Career development paths, knowledge-sharing initiatives, language classes, and sponsored training or conferences including partnerships with Databricks and Anthropic.
- Team-building events and integration budget.
- Celebrate work anniversaries, birthdays, and milestones.
- Access to medical and sports packages, eye care, and well-being support including psychotherapy and coaching.
- Full work equipment including laptop and devices.
- Opportunities to boost personal brand via conferences, blogging, and meetups.
- Smooth onboarding with a dedicated buddy and autonomous culture.
Inne informacje
Informujemy, że administratorem danych jest Addepto sp. z o.o. z siedzibą w Warszawie, ul. Swieradowska 47, 02-662. Masz prawo do żądania dostępu do swoich danych osobowych, ich sprostowania, usunięcia lub ograniczenia przetwarzania, prawo do wniesienia sprzeciwu wobec przetwarzania oraz prawo do przenoszenia danych i wniesienia skargi. Dane osobowe przetwarzane są do realizacji procesu rekrutacji. Podanie danych jest obowiązkowe w zakresie wynikającym z Kodeksu pracy. Dane będą przetwarzane do zakończenia rekrutacji i okresu dochodzenia roszczeń, a przy wyrażeniu zgody, do jej wycofania. Administrator przetwarza dane obowiązkowe na podstawie prawa, dane dodatkowe na podstawie zgody. Odbiorcą danych jest Just Join IT i podmioty przetwarzające dane w ramach rekrutacji.
Addepto
92 aktywne oferty