Lead Data Engineer (Databricks)

Godel Technologies Europe is a specialist in delivering agile at scale, building mission critical products for leading UK brands. Headquartered in Manchester, UK, with 1.5k+ employees across Europe and 1.2k in Poland across 5 offices including Bialystok. Clients include SHELL Energy, the Gym Group, Secret Escapes, ao.com, DealTrak, theidol.com, among others.

Must have:

• 7+ years of experience in Data Engineering (including minimum 5 years with Databricks)
• Strong experience in Databricks Data Platform for distributed data processing
• Excellent programming skills in Python and SQL
• Strong understanding of data modeling, data lakehouse architecture, and ELT/ETL patterns
• Experience designing scalable cloud-based data platforms (AWS / Azure)
• Knowledge of data governance, security, and access control best practices (Unity Catalog, dbt)
• Experience leading or mentoring engineers is a strong advantage
• Strong analytical thinking and problem-solving skills
• Excellent communication and collaboration skills
• Fluency in English (at least B2)

Nice to have:

• Data Streaming: Kafka
• Databases: MS SQL (SSIS, SSAS), PostgreSQL, MySQL
• BI tools: PowerBI

• Design and implement scalable data platforms and pipelines using Apache Spark on Databricks
• Lead the development of distributed data processing pipelines using PySpark and SparkSQL
• Build and manage Databricks Workflows for orchestration, scheduling, monitoring, and error handling
• Optimize Spark workloads by applying join strategies, shuffle optimization, caching, and partitioning techniques
• Design and maintain Delta Lake architectures, including schema evolution, ACID transactions, and performance tuning
• Implement data governance and access control using Unity Catalog, including permissions, lineage, and secure data sharing
• Collaborate with architects and engineering teams to design cloud-native data platforms
• Ensure data quality, observability, and reliability across pipelines and data products
• Lead performance optimization of large-scale data processing workloads
• Mentor and support other Data Engineers, contributing to engineering standards and best practices
• Participate in architecture discussions and contribute to the evolution of the company’s data engineering practices

This is a hybrid role requiring occasional office presence, especially during client visits or important company meetings. Occasional short business trips to Warsaw (approximately four times a year) are expected.