Mindbox

Spark / Data Lakehouse Developer

115 - 135 PLN/ dzień./ dzień.B2BB2B (netto)

MidHybryda·WrocławFull-time·B2B

#457511·Dodano 22 dni temu·23

Źródło: nofluffjobs.com

Aplikuj teraz

Tech Stack / Keywords

Spark Big DataCloudApache Spark ETL MongoDBData LakeApache Python ScalaData modelingCloud platformAzure AWS GCP CI/CD Git KafkaTestingdbtSecurityAudit

Firma i stanowisko

Mindbox is a tech-driven company connecting top IT talents with technology projects for leading enterprises across Europe. The role is part of the Data Platform team focused on designing, developing, and implementing data reconciliation solutions within the Operational Data Store (ODS) using MongoDB and modern Data Lakehouse architectures.

Wymagania

Proven experience as a Data Engineer or Spark Developer in big data or cloud ecosystems.
Strong knowledge of Apache Spark especially in data processing and ETL pipeline development.
Hands-on experience with MongoDB and ODS-type data store architectures.
Practical experience in Data Lakehouse technologies such as Delta Lake, Apache Iceberg, or Hudi.
Proficiency in Python and/or Scala including data manipulation and pipeline scripting.
Solid understanding of data modeling, data reconciliation, and data governance concepts.
Experience working with distributed data environments and cloud platforms (Azure, AWS, or GCP).
Familiarity with CI/CD processes and version control (Git).

Nice to have:

Experience with streaming frameworks like Kafka, Structured Streaming.
Knowledge of data quality frameworks and testing solutions such as Great Expectations, dbt tests.
Understanding of metadata management and Data Catalog tools.
Familiarity with data security, access control, and audit mechanisms.

Obowiązki

Design and implement data reconciliation mechanisms within the Operational Data Store (ODS) layer.
Develop integration and synchronization logic between ODS and source systems based on MongoDB.
Build, optimize, and maintain data pipelines using Apache Spark and distributed processing frameworks.
Design and develop data layers in Lakehouse architecture (from Raw through Curated to Gold zones).
Implement and maintain solutions using Delta Lake, Apache Iceberg, or Apache Hudi.
Ensure data quality, reliability, and monitoring across pipelines and storage layers.
Collaborate closely with data engineers, analysts, and architects to deliver end-to-end data solutions.
Participate in continuous improvement of the data platform architecture and engineering standards.

Benefity

Flexible cooperation model.
Hybrid work setup with remote days available depending on client arrangements (Warszawa or Wrocław).
Collaborative team culture with experienced professionals eager to share knowledge.
Continuous development with access to training platforms and growth opportunities.
Comprehensive benefits including Interpolska Health Care, Multisport card, Warta Insurance, and more.
High quality equipment provided including laptop and essential software.

Elastyczne godziny

Opieka zdrowotna

Karta sportowa

Ubezpieczenie