Data Engineer — Data Lakehouse

CommIT is hiring a Data Engineer in Kraków, Poland, focusing on regulated iGaming wallet and ledger data. The role involves owning the company's data lake, managing millions of financial events daily, ensuring accuracy and compliance with audit requirements.

• Minimum 3 years experience in hands-on delivery of data lakehouse architectures including pipelines, ingestion, models, and monitoring.
• Expertise in lakehouse architecture layers (bronze/silver/gold), open table formats such as Iceberg, Delta, or Hudi, and schema evolution.
• Proficiency in data layout and query optimization at terabyte scale, including partitioning, compaction, file sizing, and query performance using Trino/Athena/Snowflake.
• Production experience with cloud lakehouse or data warehouse platforms such as Snowflake, Databricks, or BigQuery.
• Strong experience with CDC and streaming ingestion technologies such as Kafka and Debezium, including handling late, duplicate, and out-of-order events.
• Strong SQL skills and data modeling knowledge sufficient to reason about OLTP systems, critical for financial ledger data.
• Ability to ensure data correctness by managing freshness SLAs, drift detection, and reconciliation with source systems.
• Knowledge of data governance including catalogs, lineage, access control, PII masking, retention policies, and GDPR compliance.
• Experience with cloud object storage such as S3 or GCS, including storage tiering and archival techniques.
• Proficiency in Python and experience with orchestration tools like Airflow or Dagster.
• Location requirement: Kraków, Poland with hybrid work model (2 days per week in office).

• Own the lakehouse architecture including bronze/silver/gold layers and Iceberg/Delta tables with schema evolution.
• Manage data ingestion via CDC streaming (using Kafka, Debezium), handling late and duplicate events.
• Design data layout optimizing for speed and cost: partitioning, compaction, file sizing, and query performance on Trino/Athena/Snowflake.
• Manage retention and archival policies including storage tiering, regulatory retention, immutability, and GDPR deletion.
• Guarantee data correctness by meeting freshness SLAs, drift detection, and reconciliation against source wallet and ledger systems.
• Oversee governance aspects including catalog and lineage, row/column access control, PII masking, encryption, and audit trails.
• Monitor ingestion health, detect data anomalies, and control cloud storage and compute expenditure.