Data Engineer

Tesco is a technology company with over 4,000 experts across 5 countries: UK, Poland, Hungary, Czech Republic, and India. Their Technology team focuses on strategic ownership of architecture, design, testing, deployment, infrastructure, operation, and security within their domains to ensure agile, smooth, and safe operations with maximum business impact.

• Experience building and optimizing data pipelines and data systems.
• Self-direction and ability to support data requirements across multiple teams, systems and products.
• Collaboration with software developers, database architects, data analysts and data scientists.
• Data processing with Spark and Scala or Python.
• Data storage with HDFS, Ceph S3 or a cloud alternative.
• Resource management with YARN or a cloud alternative.
• Lakehouse technologies: Hive or Kyuubi, or an alternative.
• Workflow scheduling with Airflow or an alternative.

Nice to have:

• Functional programming.

• Apache Kafka.

• Kubernetes.

• Stream processing.

• CI/CD.

• Minimum 2 years of experience in a similar position.

• Knowledge of Spark, Scala, Python, Airflow, Hive.

• Familiarity with HDFS, Ceph S3, Yarn, Kyuubi.

• Design, build and maintain scalable data pipeline architecture for order, delivery, customer, supplier and product data.
• Assemble complex datasets that meet functional and non-functional requirements.
• Automate manual processes and redesign infrastructure to improve data delivery and scalability.
• Build extraction, transformation and loading infrastructure for diverse data sources.
• Create analytics tools supporting customer acquisition, operational efficiency and business performance metrics.
• Support stakeholders and cross-functional teams with data issues and infrastructure needs.
• Keep data separated, secure and reliable, and build tools for analysts and data scientists.
• Collaborate with developers, database architects, analysts and data scientists.