Podsumowanie oferty
The role involves designing and developing a hybrid Kafka architecture integrating on-premise and AWS clusters. The engineer will work in a Data Platform team to build and maintain Kafka environments with Confluent Platform and AWS services. Responsibilities include data replication, CI/CD implementation, infrastructure automation, and monitoring setup. This long-term project offers remote work with occasional on-site presence in Wrocław.
Wymagania
- Minimum 4 years of experience with Apache Kafka
- Strong knowledge of Confluent Platform (Enterprise) and Confluent Cloud
- Experience in Kafka cluster migration and replication (especially MirrorMaker 2, Cluster Linking, Confluent Replicator)
- Practical experience with AWS services including VPC, IAM, EC2, EKS, MSK, S3, KMS, Secrets Manager
- Experience building and maintaining hybrid on-premise to AWS environments
- Proficiency with Terraform (module creation, versioning, plan testing; Terratest and tfsec are a plus) and Ansible
- Experience with CI/CD tools such as GitHub Actions, GitLab CI, or Jenkins
- Programming and scripting skills in Python, Ansible, and Bash
- Experience in automating integration tests for Kafka
- Knowledge of monitoring and observability tools (Prometheus, Grafana, Confluent Control Center, OpenTelemetry, Elastic Stack)
- Understanding of Kafka security fundamentals: TLS, SASL, ACL, RBAC, data encryption at rest and in transit
Obowiązki
- Design and develop hybrid Kafka architecture connecting on-premise and AWS environments
- Select and configure components of Confluent Platform including Schema Registry, Flink, Control Center, and Tableflow
- Build and configure Kafka environment in AWS (MSK, EKS, related services)
- Implement data replication and synchronization between on-prem and AWS clusters
- Configure and maintain MirrorMaker 2, Cluster Linking, and Confluent Replicator
- Ensure data consistency and low latency during replication
- Design and implement CI/CD pipelines for Kafka, schemas, and consumer applications
- Develop and maintain Infrastructure-as-Code using Terraform
- Automate deployments, tests, and infrastructure
- Implement and maintain monitoring and observability solutions using Prometheus, Grafana, Control Center, OpenTelemetry, Elastic Stack
- Create runbooks, alerts, and SLA/SLO documentation
- Troubleshoot technical incidents during solution development and maintenance
- Optimize Kafka performance and configuration tuning
- Collaborate with Data Platform, Data Engineering, DevOps, and Security teams
- Mentor less experienced engineers and conduct technical workshops
Benefity
- High flexibility for remote work
- Long-term and stable employment
- Opportunity to participate in key projects for a large company in a stable sector
- Attractive salary commensurate with skills and experience
- Benefits including Lux Med health care and MultiSport subscription
Inne informacje
Data controller is IT Leasing Team located in Warsaw at ul.Taśmowa 7. Personal data are processed for recruitment purposes under legal and consent bases. Applicants have rights to access, rectify, delete, and oppose processing, as well as to data portability and complaint filing. Providing mandatory data per labor code is required. The processing period covers recruitment and claim periods, or until withdrawal of consent for future recruitments. Data recipients include Just Join IT and other processing entities.
ITLT
135 aktywnych ofert