Data engineer

Our client is a leader in the insurance industry, focusing on building and scaling an IoT data platform that transforms data from devices and sensors into business insights.

• Minimum 4 years of commercial experience as a Data Engineer or similar role.
• Commercial experience with AWS, Spark, Python, and SQL.
• Practical knowledge of Data Mesh architecture and solutions for large-scale data processing.
• Experience building scalable and high-performance data engineering solutions in cloud-native environments.
• Practical experience with Amazon SageMaker for preparing environments and data for AI/ML solutions.
• Experience with orchestration and automation tools, especially AWS Step Functions.
• Knowledge of AWS Glue, including creating and maintaining Glue Jobs and managing data in Glue Catalog.

Nice to have:

• Experience with Terraform for infrastructure as code.
• Experience with Azure cloud and Azure DevOps.
• Familiarity with AI/ML tools and solutions.

• Design, develop, and maintain efficient data pipelines (ETL/ELT) using Spark, Python, and AWS services.
• Build and evolve a modern data platform supporting various client products and business areas.
• Implement and develop workflow orchestration and automation of CI/CD processes using AWS Step Functions.
• Integrate and process data from diverse external sources including IoT, OEM, and partner systems.
• Support ML teams by preparing and delivering data for SageMaker-based solutions.
• Optimize data processing performance and infrastructure costs in a cloud environment.
• Ensure data quality through validation, monitoring, and observability.