B2Bnetwork

Senior Data Engineer – Scala / Spark / Hadoop (f/m)

Brak informacji o wynagrodzeniu

SeniorHybryda·WarszawaFull-time·B2B

#428532·Dodano dwa miesiące temu·5

Źródło: justjoin.it

Aplikuj teraz

Tech Stack / Keywords

Hadoop and Apache SparkSQLApache Oozie and Apache AirflowScala Apache Kafka

Podsumowanie oferty

The Senior Data Engineer will design, develop, and maintain scalable data engineering solutions including pipelines and reporting processes for a complex Big Data platform. The role involves working with the financial sector's strategic regulatory initiative as part of an international team. Key responsibilities include developing workflows, optimizing SQL queries, and supporting production environments.

Wymagania

Strong hands-on experience with Scala.
Commercial experience with Apache Hadoop.
Practical knowledge of Apache Spark and large-scale data processing.
Experience building and maintaining Data Engineering pipelines.
Experience with Apache Kafka and event-driven architecture.
Practical knowledge of CI/CD processes, particularly Jenkins.
Very good knowledge of Git, code review and pull request processes.
Strong SQL development and optimisation skills.
Experience supporting production environments.
Ability to independently investigate complex technical issues.
English proficiency sufficient for daily work in an international team.
Availability to work from the Warsaw or Gdańsk office twice per week.
Ability to perform the work from Poland.

Nice to have:

Kubernetes experience.
Knowledge of Apache Flink.
Experience with Apache Airflow or Apache Oozie.
Previous involvement in regulatory or reporting projects.
Experience in the financial sector.

Obowiązki

Design, develop and maintain Data Engineering solutions using Hadoop and Apache Spark.
Build scalable solutions for processing large volumes of data.
Analyse complex business requirements and translate them into effective technical solutions.
Develop and optimise SQL queries and data-processing workflows.
Build and enhance data pipelines and reporting processes.
Create and maintain workflows using Apache Oozie and Apache Airflow.
Develop data solutions using Scala.
Work with event-driven architecture based on Apache Kafka.
Maintain and improve CI/CD processes using Git, pull requests and Jenkins.
Provide L3 support for production environments.
Investigate production incidents and technical problems in cooperation with relevant operational teams.
Collaborate with business stakeholders to define requirements and analyse issues.
Ensure the quality, performance, security and reliability of data solutions.

Benefity

Participation in a strategic regulatory initiative with a tangible impact on the financial sector.
Work within a large and advanced Big Data environment.
A modern technology stack including Scala, Hadoop, Spark, Kafka, Airflow and Jenkins.
A high level of independence and influence over technical solutions.
Cooperation with an experienced international team.
Stable, long-term cooperation.
A professional environment focused on quality, security and expert development.
An opportunity to gain experience in financial regulations and regulatory reporting.

Inne informacje

Availability to work from the Warsaw or Gdańsk office twice per week and ability to perform the work from Poland.

B2Bnetwork

171 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz