Senior/Lead Data Software Engineer (Python, Spark, Azure)

EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.

• Proficiency in Python and Spark with at least 3 years in data engineering roles
• Strong experience with Azure Databricks and PySpark
• Proven expertise in designing and implementing ETL/ELT solutions
• Experience migrating big data platforms to Azure-native services
• Proficiency with Delta tables for model tuning
• Knowledge of data governance and regulatory compliance frameworks
• Familiarity with Docker, Kubernetes (AKS), and Terraform for infrastructure automation
• Ability to manage large data volumes with high efficiency
• Excellent problem-solving and analytical skills
• Strong communication and collaboration abilities
• English proficiency at B2 level or higher

• Migrate and optimize over 500 data jobs using Azure Databricks optimization techniques
• Manage and process 12 TB of data efficiently across platforms
• Tune machine learning models for Azure environments using Java Spark and Delta tables
• Update and maintain libraries to address security vulnerabilities
• Develop and maintain ETL/ELT pipelines using PySpark and related technologies
• Collaborate with cross-functional teams to integrate GenAI capabilities into data workflows
• Monitor data quality and implement improvements to ensure accuracy and reliability
• Automate deployment and operational tasks using Terraform and GitLab CI/CD
• Support data governance initiatives to comply with regulatory standards
• Troubleshoot and resolve performance issues in data processing systems
• Document system processes and provide technical guidance to junior engineers
• Implement best practices for code quality and data security
• Participate in code reviews and knowledge sharing sessions
• Optimize costs associated with data storage and processing