Databricks Data Engineer

Devapo is a software development company specializing in complex strategic projects. They develop custom software solutions that boost ROI, from initial idea through maintenance and testing. Their services include smart process automation, custom software development, legacy system modernization, IT project outsourcing, and product design. Devapo uses modern technologies and agile methodologies to ensure quality, transparency, and customer satisfaction.

• Minimum 3 years of experience in a Data Engineer role or similar position
• Strong knowledge of Azure data services (Azure Data Factory, Azure Synapse Analytics, Azure Data Lake)
• Extensive experience with Databricks and Apache Spark
• Proficiency in SQL and experience with relational databases
• Experience in creating and optimizing data pipelines (ETL/ELT) in cloud environments
• Strong programming skills in Python, Scala, or Java
• Experience with Delta Lake architecture and optimization techniques
• Knowledge of data modeling and schema design

Nice to have:

• Experience with Azure DevOps and CI/CD pipelines for data solutions
• Knowledge of Databricks Delta Live Tables and Databricks workflows
• Experience with real-time data processing using Azure Event Hubs or Kafka
• Familiarity with Azure Purview or other data governance tools
• Databricks or Azure certifications
• Experience with Power BI or other visualization tools
• Knowledge of infrastructure as code (Terraform, ARM templates)

• Design, implement, and maintain scalable and efficient data pipelines in Azure and Databricks
• Develop and optimize ETL processes using Azure Data Factory and Databricks
• Build Big Data solutions aligned with business requirements
• Collaborate with Data Science, BI, and development teams to ensure high-quality data
• Monitor performance of data processing systems and implement improvements
• Implement data security standards and practices
• Research and evaluate new technologies in the data engineering space