Data Engineer Microsoft Fabric

Astek is a global engineering and IT consulting group founded in 1988 in France. The group supports international clients in product and service development and digital transformation across diverse industrial and technological sectors. Astek employs over 10,000 specialists working on various engineering and technology projects worldwide.

• Minimum 3 years of experience as a Data Engineer in production projects.
• Proficiency in Python, PySpark, and SQL for working with large datasets.
• Practical experience with cloud data processing platforms: Microsoft Fabric, Azure Synapse, or Databricks.
• Understanding and application of layered (medallion) data architecture.
• Familiarity with data formats such as Delta Lake and Parquet.
• Experience integrating various data sources including Excel files from business users.
• Proficient use of version control with Git in team environments.
• Willingness to sign an NDA and undergo security verification.

Nice to have:

• Microsoft certifications DP-600 or DP-700.
• Experience with Data Quality frameworks.
• Practice in creating CI/CD processes for data platforms.
• Knowledge of R language.
• Experience in the financial sector or environments with strict audit requirements.
• Familiarity with Agile and Scrum methodologies.

• Designing and implementing pipelines in Azure Data Factory and PySpark notebooks within Microsoft Fabric.
• Onboarding file and database sources with technical validation and format standardization.
• Performing data transformations and batch data preparations.
• Implementing data quality validation rules and error reporting mechanisms.
• Versioning datasets and recording metadata for processes, parameters, and sources.
• Maintaining code quality using Git, managing repositories, CI/CD deployments, and participating in code reviews.
• Collaborating closely with the Risk Team to define data transformation rules.
• Ensuring compliance with privacy and data security regulations.
• Integrating company security standards within the Azure environment.
• Optimizing processes for performance and cost efficiency.

Candidates must be willing to sign an NDA and pass a security verification, which are mandatory requirements for this position.