Caliente Interactive: Data Engineer — Data Lakehouse Senior

The role involves ownership of a company data lakehouse system handling millions of financial events daily within the regulated iGaming domain, including wallet and ledger data. The data lakehouse serves various stakeholders such as regulators, finance, and analytics, ensuring accurate and reconciled data.

• 5+ years in data engineering with real ownership of a large-scale data lake or lakehouse.
• Experience with lakehouse architecture covering bronze/silver/gold layering and open table formats like Iceberg, Delta, or Hudi, including schema evolution.
• Expertise in data layout and query optimization at terabyte scale involving partitioning, compaction, file sizing, and query performance on Trino, Athena, or Snowflake.
• Production experience with cloud lakehouse/data warehouse platforms such as Snowflake, Databricks, or BigQuery.
• Knowledge of CDC and streaming ingestion technologies like Kafka paired with Debezium or equivalents, handling late, duplicate, and out-of-order events.
• Strong SQL skills and data modeling abilities with understanding of OLTP systems, crucial for financial ledger data.
• Capabilities in maintaining data correctness including freshness SLAs, drift detection, and reconciliation.
• Governance experience including catalogs, lineage, access controls, PII masking, retention policies, and GDPR deletion.
• Familiarity with cloud object storage services such as S3 or GCS, including storage tiering and archival.
• Proficiency in Python and workflow orchestration tools like Airflow or Dagster.

Location & work model: Kraków, Poland with hybrid work requiring 2 days per week in-office.

• Own the lakehouse architecture including bronze, silver, and gold layers with Iceberg/Delta tables and schema evolution.
• Land operational data via Change Data Capture streaming using Kafka and Debezium, managing late and duplicate events.
• Design data layout optimizing speed and cost through partitioning, compaction, file sizing, and query performance on Trino, Athena, and Snowflake.
• Manage retention and archival strategies including storage tiering, regulatory retention, immutability, and GDPR deletion.
• Guarantee data correctness through freshness SLAs, drift detection, and reconciliation against source wallet and ledger systems.
• Own governance aspects such as catalog and lineage, row/column access control, PII masking, encryption, and audit trails.
• Monitor ingestion health, data anomalies, and cloud storage and compute costs.