Lead Data Platform Engineer – Apache Iceberg & PySpark

This role is for a global financial services organization specializing in Investment Banking technology solutions.

• 7+ years of experience in data engineering or related roles.
• Strong hands-on expertise with PySpark including DataFrame APIs, performance tuning, and job optimization.
• Practical experience with Apache Iceberg, especially optimizing file layout, partitioning, and internals.
• Ability to automate maintenance and operate production data workloads independently.
• Strong software engineering fundamentals focused on maintainable and reliable systems.
• Fluent English communication skills.
• Ownership mindset with focus on end-to-end delivery and initiative.

Nice to have:

• Certifications or additional skills in Big Data, Cloud platforms, or related areas.

• Lead design, development, and implementation of data pipelines and table management workflows using PySpark and Apache Iceberg.
• Drive performance and cost optimization by refining partitioning strategies, file sizing, and query efficiency.
• Manage and improve maintenance tasks including snapshot management, compaction, and metadata cleanup.
• Troubleshoot platform issues with deep understanding of Apache Iceberg internals.
• Promote engineering excellence through code reviews, automation, CI/CD, and observability best practices.
• Support operational reliability via on-call duties, incident response, and root cause analysis.
• Collaborate with cross-functional teams, influence architectural decisions, mentor engineers.
• Partner with stakeholders to ensure secure, scalable, and compliant solutions.

Only candidates with a legal right to work in the European Union will be considered for this role.