[C3F] Data Platform Engineer

Software Mind develops solutions that make an impact for companies around the globe, working on transformative projects with global clients.

• Solid commercial experience as a data or platform engineer with streaming or CDC pipelines
• Practical knowledge of Kafka including topics, partitions, offsets, consumer groups, delivery semantics, and running Kafka Connect deployments
• Strong SQL and PostgreSQL skills with knowledge of WAL, logical replication, replication slots, and lag
• Understanding of CDC concepts: snapshots, inserts, updates, deletes, event ordering, delivery, and schema evolution
• Confident Python skills for automation, orchestration, monitoring, recovery, and testing
• Hands-on experience with Azure data services such as Event Hubs, ADLS, or Azure PostgreSQL
• Comfortable using Kubernetes (deploying workloads, configuring, secrets management, log reading, pod debugging)
• Ability to debug pipeline issues from metrics and logs

Nice to have:

• Production experience with Debezium including snapshot strategies, schema history recovery, and fault recovery
• Operating stateful workloads on AKS with StatefulSets and resource tuning
• Infrastructure-as-code and CI/CD for data platform components using Terraform, Bicep, or similar
• Hands-on experience with Databricks and Azure Data Factory at scale
• Implementing data protection controls such as masking, hashing, access control, and retention policies

• Build and operate change-data-capture pipelines from PostgreSQL into Azure, using Kafka Connect and Debezium
• Configure, deploy, and scale connectors, managing setup, tasks, offsets, schema history, and snapshot strategy
• Run pipelines as stateful workloads on Kubernetes (AKS), handling configuration, secrets, networking, and resource tuning
• Monitor and troubleshoot production platform issues such as connector failures, task rebalances, restarts, throughput, backpressure, retries, and recovery
• Automate platform operations in Python including onboarding new data sources, orchestration, monitoring, alerting, recovery, and testing
• Integrate CDC streams with Azure Event Hubs, ADLS, PostgreSQL, ADF, and Databricks
• Manage platform infrastructure as code for reproducibility and reviewability
• Apply data protection measures including masking, hashing, access control, and retention