Data Platform Engineer
120 - 145 PLN/ godz.
NieokreśloneFull-time
#362508·Dodano 21 dni temu·1
Źródło: LinkGroup🚫Oferta wygasła. Ta oferta pracy nie jest już aktywna i rekrutacja została zakończona.
Tech Stack / Keywords
AzureKafkaApacheETLBigQuerySQLPythonSpark
Wymagania
- Strong data engineering background: SQL, Python, Spark/Beam, streaming systems (Kafka/Event Hubs), BigQuery and/or Azure Synapse/Databricks.
- Experience with Avro, Schema Registry, and automated SDK generation.
- Familiarity with Kubernetes, GitOps, CI/CD, and Infrastructure as Code tools (Terraform or Bicep).
Obowiązki
- Design, build, and operate streaming ingestion pipelines (Event Bus → Azure Event Hubs or Kafka) and real-time processing using frameworks such as Apache Beam or Azure Stream Analytics, as well as batch ETL pipelines into BigQuery or Azure Synapse.
- Implement schema-to-pipeline automation, including SDK generation from Avro/Schema Registry and support for schema evolution.
- Embed data quality, lineage, and governance capabilities into onboarding workflows and data pipelines.
- Expose data pipelines as self-service templates within the developer platform.
linkgroup
341 aktywnych ofert