Data Engineer (Spark)
Tech Stack / Keywords
Firma i stanowisko
Addepto is a leading AI consulting and data engineering company working with large enterprises and startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. The company specializes in Artificial Intelligence and Big Data, providing scalable, ROI-focused AI solutions. It is part of KMS Technology, a US-based global technology group, enabling enterprise-scale delivery from AI experimentation to production impact. Addepto has developed its own product, ContextClue, and contributes open-source solutions to the AI community.
Wymagania
- At least 3 years of commercial experience implementing, developing, or maintaining Big Data systems, data governance, and management processes.
- Strong programming skills in Python, Java, or Scala, including clean code practices and OOP design.
- Hands-on experience with Big Data technologies such as Spark, Cloudera, Airflow, Kafka, NiFi, Docker, and Iceberg.
- Excellent understanding of dimensional data and data modeling techniques.
- Experience in implementing and deploying solutions in cloud environments.
- Consulting experience with strong communication and client management skills.
- Ability to work independently and take ownership of project deliverables.
- Fluent in English at C1 level.
- Bachelor's degree in technical or mathematical studies.
Nice to have:
- Experience with MLOps frameworks like Kubeflow or MLFlow.
- Familiarity with Databricks and/or dbt.
Obowiązki
- Develop and maintain a high-performance data processing platform for automotive data, ensuring scalability and reliability.
- Design and implement data pipelines that process large volumes of data in streaming and batch modes.
- Optimize data workflows using technologies such as Spark, Cloudera, and Airflow.
- Manage structured and unstructured data efficiently with data lake technologies like Iceberg.
- Collaborate with cross-functional teams for seamless data integration.
- Monitor and troubleshoot the platform to ensure high availability and performance.
- Leverage cloud services (AWS) for infrastructure management and scaling workloads.
- Write and maintain high-quality Python, Java, or Scala code for data processing and automation.
Benefity
- Work with a supportive team of AI & Big Data enthusiasts.
- Engage with global enterprises and startups on international projects.
- Flexible work arrangements including remote work or modern offices and coworking spaces.
- Professional growth opportunities through career paths, knowledge-sharing, language classes, and sponsored training and conferences.
- Partnership with Databricks and Anthropic providing industry-leading training materials and certifications.
- Participation in team-building events and integration budget.
- Celebration of work anniversaries, birthdays, and milestones.
- Access to medical and sports packages, eye care, and well-being support including psychotherapy and coaching.
- Full work equipment including laptop and necessary devices.
- Opportunities to boost personal brand via conferences, blogging, and meetups.
- Smooth onboarding with a dedicated buddy in a supportive and autonomous culture.
Inne informacje
Informujemy, że administratorem danych jest Addepto sp. z o.o. z siedzibą w Warszawie, ul. Swieradowska 47, 02-662. Masz prawo do żądania dostępu, sprostowania, usunięcia, ograniczenia przetwarzania swoich danych osobowych, prawo do wniesienia sprzeciwu, przenoszenia danych oraz wniesienia skargi do organu nadzorczego. Dane będą przetwarzane w celu realizacji procesu rekrutacji. Podanie danych obowiązkowych jest wynikające z Kodeksu pracy i jest obowiązkowe. Odmowa podania danych obowiązkowych może skutkować brakiem możliwości przeprowadzenia rekrutacji. Dane będą przetwarzane do zakończenia postępowania rekrutacyjnego i okresu dochodzenia roszczeń, a w przypadku zgody na udział w przyszłych postępowaniach - do czasu wycofania zgody.
Addepto
90 aktywnych ofert