Data Platform Engineer
120 - 145 PLN/ godz.B2B
SeniorFull-time·B2B
#362350·Dodano 16 dni temu·2
Źródło: nofluffjobs.comTech Stack / Keywords
Data engineeringSQLPythonSparkKafkaBigQueryAzure SynapseSDKKubernetesInfrastructure as CodeTerraformBicep
Wymagania
- Strong data engineering background with SQL, Python, Spark/Beam, streaming systems (Kafka/Event Hubs), BigQuery and/or Azure Synapse/Databricks.
- Experience with Avro, Schema Registry, and automated SDK generation.
- Familiarity with Kubernetes, GitOps, CI/CD, and Infrastructure as Code tools such as Terraform or Bicep.
Obowiązki
- Design, build, and operate streaming ingestion pipelines (Event Bus → Azure Event Hubs or Kafka) and real-time processing using frameworks such as Apache Beam or Azure Stream Analytics, as well as batch ETL pipelines into BigQuery or Azure Synapse.
- Implement schema-to-pipeline automation, including SDK generation from Avro/Schema Registry and support for schema evolution.
- Embed data quality, lineage, and governance capabilities into onboarding workflows and data pipelines.
- Expose data pipelines as self-service templates within the developer platform.
linkgroup
378 aktywnych ofert