Senior Data Engineer

AVENGA is recruiting for a Senior Data Engineer to join an innovative data team focusing on financial services data for a captive finance company like Volvo Financial Services. The role involves working with cloud and big data technologies and ensuring data security in accordance with the Volvo Group’s infrastructure and policies.

• Bachelor’s degree in Computer Science, Engineering, or related field; Master’s degree preferred.
• 5+ years of experience in data engineering or related field, focusing on ETL processes and cloud data tools.
• Certified Azure Data Engineer and Databricks Certified Data Engineer.
• Proficiency in programming languages: Python, Java, or Scala.
• Experience with MS Cloud SQL Server, Azure Data Factory, Azure Databricks, PySpark, Spark SQL, ADLS, SQL, PowerBI, SSIS, SSRS.
• Strong knowledge of SQL and experience with relational databases.
• Familiarity with big data technologies and data processing frameworks.
• Excellent problem-solving skills, strong communication, and a collaborative mindset.

• Extract, combine, transform, load, manipulate, and cleanse data from various sources to ensure high-quality datasets for analysis, including data migration from legacy systems.
• Collaborate closely with data modelers and business analysts to understand data models and source-to-staging data mapping.
• Create and integrate data structures suitable for analysis, including developing DDLs for MS Cloud SQL Server tables and views.
• Design and develop Databricks notebooks for data extraction and transformation, creating Shared File Formats (SFFs).
• Configure Ingestion Framework (IF) tables for loading staging and Data Lake Core (DLC) tables.
• Develop and deploy algorithms in performance-efficient code across analytical and data platforms.
• Build large-scale batch and real-time data pipelines using data processing frameworks in cloud platforms or on-premises.
• Leverage continuous integration and delivery best practices to enhance data workflows.
• Track data consumption patterns, collaborate with data governance and information stewards, and participate in vetting/promoting data catalog content.
• Recommend and implement automation in integration flows to improve efficiency.
• Drive improvements in data acquisition, testing, and tooling to enhance data quality.
• Work in cross-functional agile teams to experiment, iterate, and deliver on new product objectives.
• Play a key role in requirement analysis and project estimates aligning with business goals.
• Monitor performance and optimize SQL queries for maximum efficiency.
• Prepare detailed design documents with prototypes and screenshots based on BRD and System Requirement Documents.
• Utilize cloud data tools such as Azure SQL, Azure Data Factory, and Azure Databricks to ingest and prepare data for analysis.