Senior Data Engineer – Python / Spark / AWS & GCP

We are looking for an experienced Senior Data Engineer / Data Platform Engineer to join an international Data team within the e-commerce and digital marketplace industry. The team is responsible for building next-generation, cloud-based data solutions capable of ingesting and processing petabytes of data across data lakes and data warehouses.

• 10+ years of experience in Data Engineering, Software Engineering, Distributed Systems, or a related field.
• Strong programming skills in Python, Java, or Scala.
• Strong knowledge of SQL and experience with SQL and NoSQL databases such as BigQuery, Teradata, MySQL, PostgreSQL, or Cassandra.
• Experience with Big Data technologies, including Hadoop, Hive, and Apache Spark.
• Hands-on experience with Airflow or another orchestration tool.
• Strong understanding of ETL, data lineage, data quality, backfills, and data pipeline troubleshooting.
• Experience developing both batch and streaming data pipelines.
• Experience across both software development and data engineering.
• Experience with AWS or GCP, particularly data processing at scale.
• Experience designing and supporting production services with strict SLAs.
• Experience with distributed applications and monitoring/logging tools such as Elasticsearch and Wavefront.
• Good understanding of testing methodologies, CI/CD, and software engineering best practices.
• Excellent written and verbal communication skills.

Nice to have:

• Scrum Master experience.
• Strong expertise in Python.
• Experience with Google Data Streams and Google Dataproc.
• Experience with Change Data Capture (CDC) technologies.
• Experience with modern data warehouse technologies such as Delta Lake.

• Design and develop high-volume batch and streaming data ingestion pipelines across AWS and GCP.
• Build and launch next-generation data ingestion and data curation platforms.
• Participate in system and data architecture discussions.
• Design scalable and high-performance distributed data solutions.
• Troubleshoot and resolve issues related to ETL, data lineage, data quality, backfills, and data pipelines.
• Design, implement, and support production services with strict SLAs.
• Collaborate with Software Engineers, Data Engineers, ML Engineers, Data Analysts, and other stakeholders.
• Provide technical guidance and mentor junior engineers.
• Contribute to best practices across software development, data engineering, testing, and CI/CD.
• Work with logging, monitoring, metrics, and alerting solutions to ensure platform reliability.