Principal Data Engineer

Simple Machines is a global, independent technology consultancy operating across Sydney, New Zealand, London, and Poland. They design and build modern data platforms, intelligent systems, and bespoke software at the intersection of Data Engineering, Software Engineering, and AI. The company partners with enterprises, scale-ups, and government to turn data into products, platforms, and decisions.

Core Engineering Strength:

• Strong Python and SQL skills
• Deep experience with Spark, modern data platforms (Databricks, Snowflake)
• Solid grasp of cloud data services (AWS or GCP)

Architecture & Design Judgement:

• Experience owning large-scale data platform architectures
• Strong data modelling skills and architectural decision-making
• Ability to balance trade-offs between performance, cost, and complexity

Data Platform Experience:

• Built and operated large-scale data pipelines in production
• Comfort with multiple storage technologies and formats

Engineering Discipline:

• Infrastructure-as-code experience (Terraform, Pulumi)
• CI/CD pipelines with tools like GitHub Actions, ArgoCD
• Data testing and quality frameworks (dbt, Great Expectations, Soda)

Delivery & Consulting Mindset:

• Experience in consulting or professional services
• Strong consulting instincts and client guidance skills
• Comfortable mentoring senior engineers and influencing culture

Lead Platform & Architecture Design:

• Own the end-to-end architecture of modern, cloud-native data platforms
• Design scalable data ecosystems using data mesh, data products, and data contracts
• Make architectural decisions across ingestion, storage, processing, and access layers
• Ensure platforms are secure, compliant, and production-grade by design

Build Modern Data Platforms:

• Design and deliver cloud-native data platforms using Databricks, Snowflake, AWS, and GCP
• Apply architectural patterns: data mesh, data products, and data contracts
• Integrate deeply with client systems for scalable data access

Develop High-Performance Pipelines:

• Build and optimise batch and real-time pipelines
• Work with streaming/event-driven technologies such as Kafka, Flink, Kinesis, Pub/Sub
• Orchestrate workflows using Airflow, Dataflow, Glue

Work at Scale:

• Process and transform large datasets using Spark and Flink
• Design systems for production performance

Own Data Storage & Performance:

• Work with relational, NoSQL, and analytical stores: Postgres, BigQuery, Snowflake, Cassandra, MongoDB
• Optimize storage formats and access patterns: Parquet, Delta, ORC, Avro

Cloud, Security & Governance:

• Implement secure, compliant data solutions with security by design
• Embed governance balancing developer velocity

Consult and Influence:

• Work with clients to shape solutions
• Translate business needs into engineering decisions
• Act as a trusted technical advisor

Technical Leadership & Quality:

• Set engineering standards and best practices
• Review designs and provide mentorship
• Improve data quality, testing, observability, and operational excellence