Data Engineer
120 - 140 PLN/ godz.B2B
MidFull-time·B2B
#450365·Dodano wczoraj·0
Źródło: nofluffjobs.comTech Stack / Keywords
PythonGCPBigQuerypub/subOracleSQLPostgreSQLETLInformatica PowerCenter
Wymagania
- Minimum 3 years of commercial experience designing and implementing data integration or processing workflows
- Practical experience with Google Cloud Platform, especially BigQuery, Cloud Storage, and Pub/Sub
- Practical knowledge of Oracle, including query creation and optimization, and working with large data volumes
- Development experience with Informatica PowerCenter enabling independent analysis and modification of ETL processes
- Knowledge of data modeling and designing data processing workflows
- Ability to optimize SQL queries for performance
- Understanding of data quality, consistency, and validation issues
- Experience preparing and conducting tests and creating technical documentation
Obowiązki
- Designing, implementing, and developing Data Engineering solutions on Google Cloud Platform, primarily with BigQuery, Cloud Storage, and Pub/Sub
- Migrating and restructuring data processing and feeding processes from Oracle/on-premise environments to GCP
- Designing and developing batch and streaming data processing pipelines, including data quality control, error handling, and retry mechanisms
- Designing data structures and models in BigQuery considering performance, scalability, and cost optimization
- Analyzing data and preparing logical and physical data models for Oracle, PostgreSQL, and GCP environments
- Creating, analyzing, and optimizing SQL queries, including execution plan analysis and bottleneck identification for large data volumes
- Analyzing existing Informatica PowerCenter processes and migrating them to target solutions in GCP
- Implementing transformation, validation, and data integration logic using SQL and Python
- Performing technical analyses, impact analyses, and dependency assessments, participating in solution specification alignment
- Preparing unit and integration tests, supporting system, performance, regression, and UAT testing
- Preparing deployment changes, deployment instructions, and supporting deployments and post-deployment verifications
- Diagnosing and resolving incidents related to data processing and quality
- Creating and updating technical documentation and working with code repositories and CI/CD processes following DevOps practices
DCG
362 aktywne oferty