Tech Stack / Keywords
Wymagania
- Minimum 2 years of commercial experience in designing and implementing data integration or processing pipelines.
- Experience corresponding to the Middle Data Engineer level.
- Practical knowledge of Google Cloud Platform, especially BigQuery, Cloud Storage, and Pub/Sub.
- Practical experience with Oracle, including creating and optimizing SQL queries and working with large data volumes.
- Experience with Informatica PowerCenter, enabling independent analysis and modification of ETL processes.
- Proficiency in SQL for analysis, optimization, and development of data processing.
- Experience with ETL/ELT processes, data migration, and system integration.
- Knowledge of Python necessary for implementing data processing and transformation logic.
- Experience in testing, documenting, and deploying data solutions.
Obowiązki
- Designing, implementing, and developing Data Engineering solutions on Google Cloud Platform, primarily using BigQuery, Cloud Storage, and Pub/Sub.
- Migrating and restructuring data processing and ingestion from Oracle / on-premise environments to GCP.
- Designing and developing batch and streaming processes with quality control, error handling, and retry mechanisms.
- Designing data structures and models in BigQuery, focusing on performance, scalability, and cost optimization.
- Analyzing data and preparing logical and physical data models for Oracle, PostgreSQL, and GCP.
- Creating, analyzing, and optimizing SQL queries, including execution plan analysis and bottleneck identification for large data volumes.
- Analyzing existing Informatica PowerCenter processes and migrating them to target solutions on GCP.
- Implementing data transformation, validation, and integration logic using SQL and Python.
- Conducting technical analyses, impact analyses, and participating in solution specification agreement.
- Preparing unit and integration tests, supporting system, performance, regression, and UAT testing.
- Preparing deployment changes, deployment instructions, and supporting deployments and post-deployment verification.
- Diagnosing and resolving incidents related to data processing and quality.
- Creating and updating technical documentation.
- Working with code repositories and CI/CD processes following the DevOps approach.
RITS
253 aktywne oferty