Tech Stack / Keywords
SQLApacheHadoopSparkCI/CDKafkaApache SparkAirflow
Wymagania
- Strong experience in data engineering and distributed data processing.
- Proficient in SQL for data transformation and reporting.
- Hands-on expertise with Apache Hadoop and Apache Spark.
- Experience with workflow orchestration tools such as Oozie and Airflow.
- Familiarity with CI/CD pipelines using Git, PR workflows, and Jenkins.
- Knowledge of Kafka and event-driven architectures.
- Good communication skills and ability to translate SQL issues for business stakeholders.
Nice to have:
- Experience with containerization and orchestration using Kubernetes.
- Familiarity with Apache Flink.
Obowiązki
- Develop data engineering solutions for regulatory reporting needs.
- Translate business requirements into executable SQL statements.
- Provide Level 3 support and collaborate with Production Assurance.
- Optimize data processing workflows using Apache Hadoop and Spark.
- Implement CI/CD pipelines for seamless deployment.
- Utilize Kafka for building event-driven architectures.
emagine
206 aktywnych ofert