Senior Databricks Engineer (Architect)

Project: Legacy Migration from SAS to Databricks

• Extensive experience with PySpark, Delta Lake, and Unity Catalog.
• Deep understanding of SAS datasets, procedures, and macros (ability to read and translate legacy code).
• Proven track record in architectural roles or leading large-scale data migrations.
• Hands-on experience with Azure (preferred) or AWS data services.
• Proficiency in Python and SQL at an advanced level.

• Lead the end-to-end migration of complex data pipelines from SAS to Databricks.
• Design scalable PySpark/Delta Lake solutions to replace legacy SAS Base/Macro code.
• Deconstruct existing SAS logic and re-engineer it into optimized, modern ETL/ELT workflows.
• Optimize Spark jobs and SQL queries within Databricks for maximum cost-efficiency and speed.
• Act as a technical bridge between legacy data owners and the modern cloud engineering team.