Data Engineer

Devire Outsourcing IT provides B2B cooperation for IT specialists working on innovative projects for leading clients. The current opportunity is with a client from the banking sector, focused on IT projects.

• Minimum 3 years of experience as a Data Engineer.
• Practical knowledge of GCP, especially BigQuery, Cloud Storage, and Pub/Sub.
• Very good knowledge of SQL and experience with Oracle and large data volumes.
• Experience designing and developing ETL/ELT, batch, and streaming processes.
• Practical knowledge of Informatica PowerCenter for analyzing and modifying ETL processes.
• Knowledge of Python in the context of data processing and integration.
• Experience working with Git and CI/CD.
• Ability to analyze and optimize data processing workflows.

• Design, implement, and develop Data Engineering processes on Google Cloud Platform.
• Build and maintain data pipelines using BigQuery, Cloud Storage, and Pub/Sub.
• Migrate and reengineer data processing and data feeding from Oracle/on-premise to GCP.
• Design batch and streaming processes with validation, error handling, and retry mechanisms.
• Design and optimize data structures in BigQuery considering performance, scalability, and cost.
• Create, analyze, and optimize SQL queries including execution plans and bottleneck identification with large data volumes.
• Analyze and rebuild existing Informatica PowerCenter processes and migrate them to GCP solutions.
• Implement data transformations, validation, and integration using SQL and Python.
• Prepare data models and analyze data for Oracle, PostgreSQL, and GCP environments.
• Conduct technical analyses, impact and dependency analyses, and participate in solution definition.
• Prepare unit and integration tests and support system, performance, regression, and UAT testing.
• Work with code repositories and manage CI/CD processes following the DevOps approach.

Nice to have:

• Experience in banking/financial sector.
• Experience migrating data and processes from Oracle/on-premise to GCP.
• Knowledge of PostgreSQL.
• Experience migrating Informatica PowerCenter processes to GCP.
• Knowledge of best practices for optimizing costs and performance in BigQuery.
• Experience creating technical documentation and supporting tests.

• Hybrid work model requiring physical presence two days per week in Warsaw office.