Lead Data Engineer (Databricks)

Godel Technologies Europe is a software engineering company headquartered in Manchester, UK, with over 1.5k employees across Europe and 1.2k in Poland across five offices. They deliver agile at scale and build mission-critical products for leading UK brands, including clients like SHELL Energy, the Gym Group, Secret Escapes, ao.com, DealTrak, and theidol.com. Their technical expertise includes software development, artificial intelligence, mobile app development, quality management, platform engineering, data engineering and analytics, UX/UI design, agile software development, product and business analysis, advisory, legacy transformation, and penetration testing.

Must have:

• 7+ years of experience in Data Engineering (including minimum 5 years with Databricks)
• Strong experience in Databricks Data Platform for distributed data processing
• Excellent programming skills in Python and SQL
• Strong understanding of data modeling, data lakehouse architecture, and ELT/ETL patterns
• Experience designing scalable cloud-based data platforms (AWS / Azure)
• Knowledge of data governance, security, and access control best practices (Unity Catalog, dbt)
• Experience leading or mentoring engineers is a strong advantage
• Strong analytical thinking and problem-solving skills
• Excellent communication and collaboration skills
• Fluency in English (at least B2)

Nice to have:

• Data Streaming: Kafka
• Databases: MS SQL (SSIS, SSAS), PostgreSQL, MySQL
• BI tools: PowerBI

• Design and implement scalable data platforms and pipelines using Apache Spark on Databricks
• Lead the development of distributed data processing pipelines using PySpark and SparkSQL
• Build and manage Databricks Workflows for orchestration, scheduling, monitoring, and error handling
• Optimize Spark workloads by applying join strategies, shuffle optimization, caching, and partitioning techniques
• Design and maintain Delta Lake architectures, including schema evolution, ACID transactions, and performance tuning
• Implement data governance and access control using Unity Catalog, including permissions, lineage, and secure data sharing
• Collaborate with architects and engineering teams to design cloud-native data platforms
• Ensure data quality, observability, and reliability across pipelines and data products
• Lead performance optimization of large-scale data processing workloads
• Mentor and support other Data Engineers, contributing to engineering standards and best practices
• Participate in architecture discussions and contribute to the evolution of the company’s data engineering practices

This is a hybrid role requiring occasional office presence, especially during client visits or important company meetings. Occasional short business trips to Warsaw (approximately four times a year) are expected.