Senior Data Engineer

170 - 200 PLN/ godz./ godz.B2BB2B (netto)

SeniorZdalnieFull-time·B2B

#478864·Dodano 3 dni temu·2

Źródło: nofluffjobs.com

Aplikuj teraz

Tech Stack / Keywords

ADFAirflow DatabricksProduct developmentData pipelinesAzure Data FactoryAzure Databricks Apache Airflow PySpark SQLData engineeringPython UNITYPower BITableauData modellingDesign PatternsData warehouseData LakeETLCommunication skills

Podsumowanie oferty

Drive the evolution of an Azure-based Data Platform and Lakehouse architecture through Data Products development. Build data pipelines and transformations using Azure Data Factory, Apache Airflow, Azure Databricks, PySpark and SQL.

Wymagania

6+ years of experience in data engineering.
Experience with Python and PySpark.
Advanced-level SQL.
Hands-on experience with an Azure-based data platform, including Databricks, Unity Catalog, Apache Airflow and Azure Data Factory.
Experience with reporting and BI tools such as Power BI and Tableau, or similar.
Solid understanding of data modelling fundamentals, main architectures and design patterns.
Knowledge of Data Warehouse, Data Lake and Data Lakehouse architectures.
Knowledge of Medallion architecture and metadata-driven architecture.
Knowledge of ETL/ELT design patterns.
Understanding of partitioning, optimization and performance, as well as batch and streaming pipelines.

Obowiązki

Implement scalable, reliable and reusable data pipelines using Azure Data Factory and Apache Airflow.
Develop data transformations in Azure Databricks using PySpark and SQL to create and extend data products following the Medallion Architecture layers: Bronze, Silver and Gold.
Optimize the performance, cost and reliability of data workloads.
Raise questions during business maturation to detect risks before the development stage.
Participate in architecture reviews and technical decision-making.
Promote engineering excellence through code reviews and design reviews.
Collaborate with Architects and Product Owners on the technical maturation of data products.
Support the Run/Operations team with complex production pipeline and data product topics.
Troubleshoot and resolve data incidents, failures and performance issues.
Participate in incident management, root cause analysis and post-mortems.
Ensure proper handover, documentation and operational readiness of developed solutions.
Contribute to improving runbooks, monitoring and operational processes.

Benefity

Sport subscription.
Training budget.
Private healthcare.

Karta sportowa

Dofinansowanie szkoleń

Opieka zdrowotna

Margo

41 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz