Cloud Data Pipelines Engineer
120 - 135 PLN/ godz.B2B
#455345·Dodano 6 dni temu·1
Źródło: nofluffjobs.comTech Stack / Keywords
PythonRCloud platformGoogle Cloud PlatformGCPMicrosoft AzureDockerKubernetesCI/CD PipelinesGitHub ActionsGitLab CISQLData warehousesPostgreSQLBigQuerySnowflakeAzure SQLGitAutomated testingAPI
Wymagania
- Advanced proficiency in Python and/or R, with strong experience writing clean, modular, and well-tested code.
- Hands-on production experience with either Google Cloud Platform (GCP) or Microsoft Azure.
- Practical experience with Docker, Kubernetes, and CI/CD pipelines (GitHub Actions, GitLab CI).
- Proficiency with SQL and relational/non-relational data warehouses such as PostgreSQL, BigQuery, Snowflake, or Azure SQL.
- Solid grasp of version control (Git), automated testing practices, and API design principles.
Obowiązki
- Containerize, deploy, and orchestrate custom R and Python scripts to run reliably in cloud-native serverless or containerized environments.
- Leverage managed cloud services on GCP (Cloud Functions, Cloud Run, BigQuery, Dataflow) or Azure (Azure Data Factory, Azure Functions, Databricks, Synapse Analytics).
- Implement and maintain workflow automation tools such as Apache Airflow or Prefect to schedule and monitor script execution.
- Implement automated logging, error handling, and testing frameworks to validate data integrity and script outputs.
- Tune data processing jobs and cloud resource allocation for cost-efficiency, execution speed, and scalability.
Link Group
514 aktywnych ofert