DataOps Engineer (Databricks + AWS)

In this role, you will join SoftServe as a mid-level DataOps Support Engineer, owning the reliability, scalability, and governance of a Databricks-based data platform. You will be responsible for operating data pipelines, ensuring data quality, and managing the cloud infrastructure that supports them. This position combines data engineering, platform operations, and DevOps practices in a fast-paced environment. The role follows a shifted schedule from 14:00 to 22:00 (Polish time), Monday to Friday.

• 2+ years of experience in data engineering or DevOps roles
• Strong proficiency in Python, PySpark, and SQL
• Hands-on experience with Databricks, including pipeline development, cluster management, Unity Catalog, and DLT orchestration
• Understanding of Delta Lake internals and structured streaming concepts such as checkpointing, watermarking, and trigger modes
• Experience with ETL/ELT data modeling techniques, including star schema and slowly changing dimensions
• Practical experience with Terraform, including module design, state management, and infrastructure governance
• Working knowledge of AWS services such as VPC, S3, IAM, CloudWatch, CloudTrail, and KMS
• Ability to analyse Spark UI query plans and resolve performance issues beyond basic optimizations
• Availability to work afternoon/evening hours (14:00–22:00 Polish time, Monday to Friday)
• Strong communication skills and ability to collaborate across cross-functional teams
• English proficiency at least at Upper-Intermediate level
• Relevant Databricks or AWS certifications are a plus

• Build, maintain, and operate data pipelines on Databricks, including Spark jobs, DLT workflows, and streaming workloads
• Troubleshoot and resolve production issues using Spark UI, query plans, AQE analysis, and cluster diagnostics
• Manage AWS cloud infrastructure, including VPC, S3, IAM, and monitoring tools
• Configure alerting systems, support incident response, and participate in operational activities
• Optimise Databricks compute resources, including cluster policies, autoscaling, SQL Warehouse configuration, and performance tuning
• Develop and maintain ETL/ELT pipelines following established data modeling practices
• Ensure data quality standards and maintain data lineage across the platform
• Manage infrastructure as code using the Databricks Terraform provider, including workspace provisioning and networking
• Maintain release and rollback processes, contribute to branching strategies, and participate in PR reviews
• Administer Databricks workspace, including access control, secrets management, and audit logging using system tables and AWS CloudTrail