Data Engineer (Databricks)

Currently, we are searching for a full-time Data Engineer to cooperate with our US client - a BioPharma organization delivering cloud software and services designed for the BioPharma sector. The mission is to unlock the potential of AI/ML to improve the lives of patients.

• Strong hands-on experience with Python and complex SQL, and production Spark experience on Databricks (Structured Streaming, Auto Loader, Delta Lake, Unity Catalog) for analytics, AI/ML, and enterprise applications
• Experience working with structured and unstructured data across the full lifecycle: ingestion, transformation, enrichment, indexing, and retention
• Proven experience ingesting from relational (Oracle, PostgreSQL) and document stores (MongoDB), using full load + CDC or equivalent replication patterns with handling schema drift, deletes, and late or out-of-order data
• Skills in data modelling across layered architecture with clear contracts between raw, curated and serving layers supporting cross-entity queries and performant data access

Nice to have:

• Experience building unstructured document pipelines including parsing and extraction from PDF, Office and scanned formats, chunking, embedding generation, and maintaining searchable indexes at scale

• Build, evolve, and operate data ingestion and processing capabilities for structured, semi-structured, and unstructured data
• Implement and maintain rich metadata and data quality practices enabling cross-record querying, traceability, and AI-ready data access
• Collaborate with architects, AI engineers, workflow engineers, and domain experts to ensure data usability, performance, and trustworthiness
• Support engineering quality through code reviews and mentoring of junior/mid-level engineers

Due to the client's location, the team works until 6:00 PM CEST.