ETL Developer with Databrics

The position is offered by eConsulting for a mostly remote role with occasional visits in Warsaw, focused on building scalable ETL/ELT data pipelines using Databricks and working within Data Lake / Lakehouse architectures on cloud platforms such as Azure.

• 3+ years experience in data engineering or ETL development.
• Hands-on experience with Databricks (PySpark, Spark SQL).
• Experience with cloud platforms, preferably Azure, with AWS or GCP acceptable.
• Knowledge of Data Lake / Lakehouse architecture.
• Strong problem-solving skills and ability to work independently.
• Experience with version control using Git.

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks (PySpark, Spark SQL).
• Integrate and transform data from multiple sources including databases, APIs, flat files, and streaming.
• Optimize data pipelines for performance, scalability, and reliability.
• Manage and structure data in Data Lake / Lakehouse environments (Bronze/Silver/Gold layers).
• Implement data quality checks and validation frameworks to ensure consistent data delivery.
• Translate business requirements into technical solutions and propose architectural improvements.
• Monitor, troubleshoot, and maintain ETL jobs to ensure smooth operation and minimal downtime.