Data Engineer (GCP experienced)

Brak informacji o wynagrodzeniu
MidFull-time
#353508·Dodano dziś·0
Źródło: nofluffjobs.com
Aplikuj teraz

Tech Stack / Keywords

SQLGCPBigQueryDataplexPythonCI/CDSparkAirflowML

Wymagania

  • Strong hands-on experience with Google Cloud Platform (GCP), including BigQuery, Dataplex, Dataflow, and Pub/Sub for building and supporting scalable data pipelines in production environments
  • Practical experience implementing and operating Dataplex for data governance, metadata management, and data quality across large-scale data ecosystems
  • Strong proficiency in Oracle SQL and PL/SQL, including complex query optimization, stored procedures, performance tuning, and working with large enterprise datasets
  • Programming experience in Python (Java/Scala as applicable) for data processing, ETL development, and pipeline automation
  • Solid understanding of data modeling and data warehouse design principles, including dimensional modeling (star/snowflake schemas) and analytics-optimized structures
  • Hands-on experience with Terraform and Infrastructure as Code (IaC) for provisioning and managing cloud resources in a reproducible and version-controlled way
  • Experience building and maintaining CI/CD pipelines, supporting automated testing, deployment, and release processes for data and platform components
  • Familiarity with Kubernetes (K8s) for deploying, scaling, and managing containerized applications in cloud environments
  • Comfortable working in Agile/Scrum teams, participating in sprint planning, code reviews, and iterative delivery cycles

Nice to have:

  • Experience with Apache Spark / Databricks for large-scale data processing
  • Knowledge of streaming architectures (Kafka, Kinesis, or Pub/Sub in advanced use cases)
  • Experience with dbt (data build tool) for transformation layer development
  • Exposure to multi-cloud environments (GCP + AWS) and hybrid data platforms
  • Knowledge of cost optimization in cloud data platforms (BigQuery partitioning, clustering, query optimization, FinOps basics)
  • Experience with Airflow / workflow orchestration tools (Apache Airflow, Cloud Composer)
  • Familiarity with BI tools integration (Looker, Tableau, Power BI)
  • Exposure to ML data pipelines / feature engineering pipelines
  • Experience with event-driven architectures and microservices integration patterns

Obowiązki

  • Designing scalable, secure pipelines across GCP (BigQuery, Dataflow, Pub/Sub) and AWS
  • Implementing governance and data domain structures using Dataplex
  • Integrating cloud platforms with Oracle databases
  • Delivering both real-time and batch data solutions
  • Optimizing systems for performance and latency
  • Automating infrastructure with Terraform
  • Building CI/CD workflows
  • Working with Docker & Kubernetes
  • Collaborating in Agile/Scrum teams
  • Ensuring governance, lineage, and compliance standards
Innowise

Innowise

24 aktywne oferty

Zobacz wszystkie oferty
Aplikuj teraz