Data Engineer (Microsoft Fabric)

Scalo delivers software projects and supports partners in business development, using a broad range of IT services including technology consulting, software development, embedded systems, cloud solutions, data management, and dedicated project teams. The cooperation model is hybrid with 1 day per week in Warsaw, focusing on software, embedded, data, and cloud service projects.

• Minimum 3 years of experience as a Data Engineer in production projects
• Practical knowledge of Python (including PySpark) and SQL working with large datasets
• Experience with cloud data processing platforms such as Microsoft Fabric, Azure Synapse, or Databricks
• Familiarity with layered data architecture (medallion or similar) and data formats Delta and Parquet
• Experience using the Git version control system
• Experience integrating diverse data sources including Excel files provided by business users
• Willingness to sign NDA and undergo a security verification process

Nice to have:

• DP-600 or DP-700 certification
• Experience in the financial sector or other highly audited environments
• Knowledge of R language
• Experience building and maintaining CI/CD pipelines for data platforms
• Experience with Data Quality frameworks
• Familiarity with Agile and Scrum methodologies

• Design and implement pipeline solutions in Data Factory and PySpark notebooks within the Microsoft Fabric environment
• Onboard file and database sources including technical validation and data format standardization
• Perform data transformations and prepare batch data
• Implement data quality validation rules and error reporting mechanisms
• Version dataset collections and metadata including parameters, sources, and versions
• Deliver services under Git version control, including code review and CI/CD deployments
• Collaborate with risk teams to define data transformation rules
• Ensure compliance with data privacy and security regulations and integrate company security standards in the Azure environment
• Optimize processes for performance and cost effectiveness
• Participate in projects within the insurance industry

Candidate must be willing to sign NDA and undergo a security verification process. Work mode is hybrid with mandatory presence in Warsaw once a week.