Staff Consultant - Data Engineer

Ready to shape the future of AI in finance? We’re looking for a talented Staff Consultant - Data Engineer to join one of the world's most renowned banking brands. Serving over 100 million customers across diverse businesses, you'll be a key player on a dynamic, remote team spanning the UK and Spain. This project governs the large-scale legacy migration and modernization of data infrastructure, converting legacy Ab Initio ETL platform to a modern, cloud-native AWS Glue and PySpark environment, leveraging advanced generative AI patterns.

Technical Expertise:

• Expert-level PySpark development and optimization.
• Deep expertise in AWS Glue and data pipeline development.
• Strong SQL and data modeling skills.
• Experience with performance tuning and cost optimization.
• Understanding of data quality frameworks and testing.

Project Experience:

• Led development of at least 2 large-scale data pipelines (1,000+ jobs or equivalent).
• Experience with ETL/ELT migrations or implementations.
• Track record of delivering on-time and within quality standards.

Leadership & Soft Skills:

• Proven ability to mentor junior engineers.
• Strong communication and documentation skills.
• Problem-solving and debugging expertise.
• Attention to detail and quality focus.
• Collaborative team player.

Data Engineering & Pipeline Development:

• Design, build, and deploy highly scalable data pipelines and modern ETL/ELT workflows using AWS Glue and PySpark.
• Lead end-to-end development, implementation, and migration of enterprise-scale data processing systems managing 1,000+ jobs.
• Architect logical and physical data models and write complex, optimized SQL queries.

Performance & Quality Assurance:

• Monitor and fine-tune cloud data infrastructure applying advanced PySpark optimization techniques.
• Implement and enforce data quality frameworks including automated testing.
• Debug complex pipeline failures and architectural bottlenecks as technical escalation point.

Leadership & Delivery:

• Provide hands-on guidance and mentorship to junior engineers.
• Drive delivery of critical data projects ensuring on-time completion and adherence to quality standards.
• Create and maintain technical documentation communicating system designs and architectural decisions.