payabl

Data Engineer

Brak informacji o wynagrodzeniu

MidHybryda·KrakówFull-time

#386841·Dodano 3 miesiące temu·3

Źródło: payabl.

Aplikuj teraz

Tech Stack / Keywords

DatabasesSQL PythonArchitectureAWS Apache Kafka PySpark

Firma i stanowisko

payabl. empowers businesses through payments innovation and banking services. It is a licensed financial company specializing in global payments and multi-currency accounts, providing a portfolio of global financial services via its platform payabl.one.

Wymagania

Minimum 3+ years of experience in data engineering or related roles.
Strong experience with SQL and data modeling for analytics and reporting.
Strong programming experience with Python, ideally including PySpark.
Experience designing and maintaining ETL/ELT pipelines in production.
Experience with real-time or near-real-time data ingestion.
Experience working with Kafka or similar streaming technologies.
Experience with CDC concepts and tools such as Debezium.
Experience with data lake or lakehouse architectures on cloud platforms.
Hands-on experience with AWS data services such as S3, Glue Catalog, Glue Jobs, EMR, IAM.
Experience with Apache Iceberg, Delta Lake, or similar open table formats.
Experience designing curated analytics layers like silver and gold layers in medallion architecture.
Experience with orchestration tools such as Apache Airflow or Dagster.
Experience working with relational databases such as MySQL, MariaDB, or PostgreSQL.
Working knowledge of Unix/Linux environments and shell scripting.
Understanding of data quality, governance, lineage, and production monitoring.

Nice to have:

Experience with Apache Druid, ClickHouse, Snowflake, Databricks, or other analytical databases.
Experience with Airbyte or similar data integration tools.
Experience with dbt or collaboration with analytics engineering teams.
Experience with Terraform and Terragrunt.
Experience with Docker and Kubernetes.
Experience optimizing Spark jobs on EMR or AWS Glue.
Experience with data visualization tools such as Tableau, Power BI, Superset, AWS QuickSight.
Experience with data observability, alerting, and monitoring tools.

Obowiązki

Lakehouse Architecture and Data Platform Design:

Design, build, and maintain scalable data lakehouse solutions on AWS using S3, Apache Iceberg, and AWS Glue Catalog.
Contribute to the evolution of the medallion architecture ensuring reliable and performant bronze, silver, and gold layers.

CDC and Streaming Data Ingestion:

Build and support real-time and near-real-time ingestion pipelines using Debezium, Kafka, Kafka Connect, and Iceberg sinks.
Monitor and troubleshoot streaming pipelines including connector issues, schema changes, and data consistency.

Data Modelling and Business-Ready Layers:

Design and implement silver and gold layer datasets transforming raw data into trusted business-ready products.
Collaborate with analysts and business stakeholders to understand requirements and create reusable data models.

Batch and Distributed Processing:

Develop and maintain batch and distributed processing jobs using PySpark on AWS EMR and AWS Glue.
Optimize data transformation jobs for performance, scalability, reliability, and cost-efficiency.

Data Integration and Orchestration:

Build and maintain data workflows with Apache Airflow for API ingestion, batch processing, and orchestration.
Support integrations from external and third-party systems using tools like Airbyte.

Data Quality, Governance, and Reliability:

Implement data quality checks, validation, and reconciliation to ensure trusted datasets.
Contribute to data governance including documentation, ownership, lineage, access, and compliance.

Infrastructure and DevOps Collaboration:

Collaborate on AWS infrastructure and deployment using services such as S3, Glue, EMR, IAM, EKS.
Support infrastructure-as-code workflows using Terraform and Terragrunt.

Benefity

Provident Fund activated after probation for future financial security.
Annual Learning Budget for professional development post-probation.
€150 monthly Wolt allowance for meals.
SportsBenefits membership providing access to gyms and sports facilities.
Eligibility for a company car after one year based on performance and availability.
Complimentary parking space near the office.
25 days of vacation plus public holidays and an additional 10 sick days.
Exclusive local discount card and tickets for events.
Free Greek language classes twice a week.
Company celebrations bringing all offices together.
Opportunities to participate in international company events and initiatives.

Karta sportowa

Opieka zdrowotna

Inne informacje

Please note if hired in Poland or Portugal, the Talent Acquisition Partner will inform about specific working arrangements and conditions during the recruitment process.

payabl

19 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz