AWS Data Engineering Lead/Solution Architect

YourITeams is hiring for an experienced Data Engineering Lead / Solution Architect to provide dual technical leadership within their platform and data engineering organization.

• Proven experience in a technical leadership or solution architect role, preferably within platform or data engineering domains.
• Strong hands-on experience with Databricks and Apache Spark; familiarity with Hive and Hadoop ecosystems.
• Deep knowledge of AWS services relevant to data platforms (e.g., EMR, S3, Glue, Lambda, EC2, RDS, IAM).
• Experience designing and operating large-scale data processing pipelines and ETL/ELT architectures.
• Proficiency in infrastructure-as-code tools and CI/CD practices for data and platform deployments.
• Demonstrable skills in performance optimisation, monitoring and fault-tolerant design for data workloads.
• Strong communication skills and the ability to explain complex technical concepts to diverse stakeholders.
• Relevant tertiary qualification or equivalent professional experience in computer science, engineering or a related discipline.

• Lead technical design and architecture for platform and data solutions, ensuring alignment with business objectives and technical standards.
• Architect, implement and optimise data processing pipelines using Databricks, Apache Spark and related technologies.
• Design and operate scalable data platforms on AWS, incorporating best practices for security, cost control and reliability.
• Collaborate with cross-functional teams (data engineers, software engineers, product managers) to translate requirements into pragmatic technical solutions.
• Provide technical leadership and mentorship, fostering engineering discipline, code quality and operational excellence.
• Define and enforce platform patterns, reusable components, and infrastructure-as-code practices.
• Drive performance tuning, capacity planning and observability for data workloads; troubleshoot critical incidents and lead root-cause analysis.
• Evaluate and integrate new technologies and open-source tools relevant to data processing and platform operations.