Nowa
Data Engineer (GCP experienced)
Brak informacji o wynagrodzeniu
MidFull-time
#353508·Dodano dziś·0
Źródło: nofluffjobs.comTech Stack / Keywords
SQLGCPBigQueryDataplexPythonCI/CDSparkAirflowML
Wymagania
- Strong hands-on experience with Google Cloud Platform (GCP), including BigQuery, Dataplex, Dataflow, and Pub/Sub for building and supporting scalable data pipelines in production environments
- Practical experience implementing and operating Dataplex for data governance, metadata management, and data quality across large-scale data ecosystems
- Strong proficiency in Oracle SQL and PL/SQL, including complex query optimization, stored procedures, performance tuning, and working with large enterprise datasets
- Programming experience in Python (Java/Scala as applicable) for data processing, ETL development, and pipeline automation
- Solid understanding of data modeling and data warehouse design principles, including dimensional modeling (star/snowflake schemas) and analytics-optimized structures
- Hands-on experience with Terraform and Infrastructure as Code (IaC) for provisioning and managing cloud resources in a reproducible and version-controlled way
- Experience building and maintaining CI/CD pipelines, supporting automated testing, deployment, and release processes for data and platform components
- Familiarity with Kubernetes (K8s) for deploying, scaling, and managing containerized applications in cloud environments
- Comfortable working in Agile/Scrum teams, participating in sprint planning, code reviews, and iterative delivery cycles
Nice to have:
- Experience with Apache Spark / Databricks for large-scale data processing
- Knowledge of streaming architectures (Kafka, Kinesis, or Pub/Sub in advanced use cases)
- Experience with dbt (data build tool) for transformation layer development
- Exposure to multi-cloud environments (GCP + AWS) and hybrid data platforms
- Knowledge of cost optimization in cloud data platforms (BigQuery partitioning, clustering, query optimization, FinOps basics)
- Experience with Airflow / workflow orchestration tools (Apache Airflow, Cloud Composer)
- Familiarity with BI tools integration (Looker, Tableau, Power BI)
- Exposure to ML data pipelines / feature engineering pipelines
- Experience with event-driven architectures and microservices integration patterns
Obowiązki
- Designing scalable, secure pipelines across GCP (BigQuery, Dataflow, Pub/Sub) and AWS
- Implementing governance and data domain structures using Dataplex
- Integrating cloud platforms with Oracle databases
- Delivering both real-time and batch data solutions
- Optimizing systems for performance and latency
- Automating infrastructure with Terraform
- Building CI/CD workflows
- Working with Docker & Kubernetes
- Collaborating in Agile/Scrum teams
- Ensuring governance, lineage, and compliance standards
Innowise
24 aktywne oferty