Senior PySpark Data Engineer – Big Data & Cloud Integration

The role is for a Kraków-based client specializing in large-scale data engineering and cloud integration, focusing on transforming experimental insights into scalable, high-performance data workflows.

• 5+ years of experience in data engineering and PySpark/Spark development.
• Strong SQL skills and experience with Spark optimization techniques.
• Hands-on experience with the Hadoop ecosystem, especially Spark on YARN.
• Knowledge of cloud platforms, preferably Google Cloud Platform (GCP), is a strong plus.

Nice to have:

• Certifications or additional experience in cloud computing or big data solutions.

• English language proficiency with communicative skills.

• Convert experimental code into scalable, efficient workflows and modules.
• Develop and optimize data pipelines using PySpark and Spark on YARN.
• Collaborate with data scientists and engineers to productionize analytical solutions.
• Ensure code quality, perform performance tuning, and implement best data engineering practices.
• Maintain and improve performance of data workflows in a large-scale environment.
• Integrate data pipelines within cloud ecosystems, primarily Google Cloud Platform (GCP).
• Support continuous improvement through performance monitoring and optimization.

Only candidates with an existing legal right to work in the European Union will be considered for this role.