Data Engineer (AWS / PostgreSQL / AWS / LLMs)

Square One Resources is hiring a Senior Data Engineer to design, build, and maintain scalable data pipelines and infrastructure supporting analytical products. The role involves delivering reliable, high-performance data systems with robust ETL/ELT processes, optimized data storage and processing, and ensuring data quality, availability, and scalability.

• Minimum 3 years of commercial experience in Data Engineering roles.
• Strong hands-on experience with Python for building data pipelines, data validation, and orchestration frameworks.
• Advanced knowledge of PostgreSQL: schema design, indexing strategies, query optimization, and performance tuning.
• Proven experience in ETL/ELT pipeline design and production-grade implementations.
• Practical experience with distributed data processing and storage technologies (e.g., AWS Athena, Apache Spark or similar).
• Strong experience with AWS services: S3, EKS, Glue, Athena.
• Experience with modern data architectures including data lakehouse patterns and ELT approaches.
• Knowledge of data warehousing modeling techniques, including dimensional modeling and reusable transformation patterns.
• Strong understanding of Git workflows, code review processes, testing strategies, and CI/CD pipelines.
• Experience with workflow orchestration tools (e.g., AWS Step Functions, Prefect, Dagster or similar).
• Experience with data quality frameworks and data observability solutions.
• Exposure to streaming or near real-time data processing (e.g., Kafka, Spark Streaming, Pub/Sub).
• Ability to use LLM-based AI agents effectively to improve engineering productivity.
• Strong systems thinking with focus on scalability, reliability, and correctness.
• Ability to balance delivery speed with maintainability, cost efficiency, and data quality.
• Strong written and verbal communication skills in collaboration with both technical and business stakeholders.
• Ability to independently own end-to-end data products, from architecture to production support.

• Design, build, and maintain scalable ETL/ELT data pipelines for ingestion, transformation, and delivery of data across the organization.
• Develop and optimize distributed data processing workflows in Python for large-scale data transformation and aggregation.
• Design, manage, and optimize PostgreSQL schemas, tables, indexes, and query performance for analytics and reporting use cases.
• Build and maintain Python-based data workflows for orchestration, validation, and reliable cross-environment data delivery.
• Implement monitoring, validation, and observability mechanisms to ensure data quality, timeliness, and completeness.
• Design and manage cloud-based data infrastructure on AWS.
• Collaborate with data analysts and business stakeholders to translate requirements into scalable and maintainable data products.
• Maintain technical documentation covering data pipelines, data models, lineage, and infrastructure components.
• Troubleshoot data pipeline issues, perform root cause analysis, and implement corrective actions.