Senior Data Engineer / Databricks Developer

The project involves building a scalable Data Quality Monitoring solution based on DQX for a healthcare/pharmaceutical client. The solution focuses on enabling business users and data owners to monitor and manage data quality across key clinical data domains, starting with Study Management data. It is built natively on Databricks and aims to improve transparency, data ownership, and audit readiness.

• Strong hands-on experience with Databricks.
• Experience with Spark, SQL, Delta Lake, and Python.
• Experience designing and implementing data pipelines / ETL-ELT.
• Experience with data quality frameworks, preferably DQX.
• Understanding of data governance, data ownership, and rule-based data quality monitoring.
• Ability to translate business requirements into scalable technical solutions.
• Experience working in complex enterprise environments.

Nice to have:

• Experience with regulated or compliance-heavy environments.
• Experience with clinical operations, trial operations, or life sciences data.
• Experience with dashboarding and business-facing data quality reporting.
• Azure DevOps / CI-CD experience.
• Experience with integrations, APIs, or downstream system connectivity.
• Snowflake experience.
• Understanding of Veeva or related clinical systems.

• Design and implementation of a Databricks-native Data Quality Monitoring framework (DQX).
• Configuration and implementation of DQX-based data quality rules.
• Development of data pipelines and data models supporting monitoring and reporting.
• Creation of dashboards and trending views for business users and data owners.
• Establish linkage between data quality rules, data usage, and critical data points.
• Documentation of technical design, rule logic, and operating model.
• Support scaling the solution to additional Clinical data domains.

Work mode is fully remote. Assignment type is B2B contract longer than 6 months with extensions. Language required is English. Recruitment process includes 2 interviews with the client.