Senior Data Engineer (Databricks)
Tech Stack / Keywords
Firma i stanowisko
Addepto is a leading AI consulting and data engineering company focused exclusively on Artificial Intelligence and Big Data. The company builds scalable, ROI-focused AI solutions for global enterprises and startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. Addepto, part of the US-based KMS Technology group, develops its own product ContextClue and contributes to open-source AI solutions.
Wymagania
- At least 5 years of commercial experience implementing, developing, or maintaining Big Data systems.
- Strong programming skills in Python, including clean code and OOP design.
- Strong SQL skills with experience in performance tuning, query optimization, and data warehousing solutions.
- Experience designing and implementing data governance and management processes.
- Deep expertise in Big Data technologies including Apache Airflow, Dagster, and Databricks.
- Experience implementing and deploying solutions in cloud environments, preferably Azure.
- Knowledge of building and deploying Power BI reports and dashboards.
- Excellent understanding of dimensional data and data modeling techniques.
- Consulting experience with ability to guide clients on architecture, technology selection, and best practices.
- Ability to work independently and own project deliverables.
- Familiarity with Spark, Azure Event Hub, and Kafka.
- Master’s or Ph.D. in Computer Science, Big Data, Mathematics, Physics, or related field.
Obowiązki
- Design and optimize scalable data processing pipelines for streaming and batch workloads using Databricks, Apache Airflow, and Dagster.
- Architect and implement end-to-end data platforms ensuring high availability, performance, and reliability.
- Lead development of CI/CD and MLOps processes for deployment automation, monitoring, and model lifecycle management.
- Develop and maintain applications for processing and analyzing diverse data sources.
- Collaborate with Data Science teams on Machine Learning projects including text/image analysis and predictive model deployment.
- Design and manage complex data transformations using Databricks, DBT, and Apache Airflow ensuring data integrity.
- Translate business requirements into scalable technical solutions with optimal performance and data quality.
- Ensure data security, compliance, and governance best practices across all pipelines.
Benefity
- Work with a supportive team passionate about AI and Big Data.
- Engage with global enterprises and cutting-edge startups on international projects.
- Flexible work arrangements with options for remote work or modern offices.
- Professional growth through career paths, knowledge-sharing, language classes, and sponsored training/conferences including partnerships with Databricks and Anthropic.
- Participate in team-building events and benefit from integration budget.
- Celebrate work anniversaries, birthdays, and milestones.
- Access medical and sports packages including eye care and well-being services.
- Receive full work equipment including laptop and necessary devices.
- Opportunities to boost personal brand through conferences, blogging, and meetups.
- Smooth onboarding with a dedicated buddy in a friendly and autonomous culture.
Inne informacje
Informujemy, że administratorem danych jest Addepto sp. z o.o. z siedzibą w Warszawie, ul. Swieradowska 47, 02-662. Masz prawo do żądania dostępu do swoich danych osobowych, ich sprostowania, usunięcia, ograniczenia przetwarzania, wniesienia sprzeciwu, przenoszenia danych oraz wniesienia skargi do organu nadzorczego. Dane są przetwarzane w celu rekrutacji, podanie wymaganych danych jest obowiązkowe, a dane będą przetwarzane do zakończenia postępowania i okresu dochodzenia roszczeń.
Addepto
41 aktywnych ofert