Codest Ltd.

Data Engineer w/ Databricks

22k - 30k PLN22 000 - 30 000 PLN/ mies./ mies.UoPUmowa o pracę (brutto)

SeniorZdalnie·WarszawaFull-time·Umowa o pracę

#456835·Dodano 15 dni temu·7

Źródło: Codest Ltd. Company No. 12590542, VAT number: GB363431020

Aplikuj teraz

Tech Stack / Keywords

Databricks Python PySpark SQL AWSGlueLambdaRedshiftS3Docker

Firma i stanowisko

We are The Codest - International Tech Software Company with tech hubs in Poland delivering global IT solutions and projects. Our expertise centers on web development, cloud engineering, DevOps and quality. We have developed a product called Yieldbird, honored as a laureate of the Top25 Deloitte awards. Our mission is to help tech companies build impactful products and scale their IT teams through boosting IT delivery performance.

Wymagania

6+ years of experience in data engineering, designing and deploying production-grade data pipelines and ETL/ELT workflows
Proficiency in Python, PySpark, and SQL
Extensive hands-on experience with AWS services such as Glue, Lambda, Redshift, and S3
Solid background in data governance and management tools including Unity Catalog and AWS Sagemaker Unified Studio
Proven experience in data migration initiatives and modernizing legacy systems
Communicative English language skills

Nice to have:

Databricks Certified Professional Data Engineer or equivalent certification
Excellent communication and teamwork skills
Strong analytical thinking and problem-solving ability
Experience with containerization and orchestration tools like Docker and Kubernetes

Obowiązki

Experience with Medallion Architecture (Bronze, Silver, Gold) for lakehouses using optimized performance patterns
Build strong data quality frameworks with automated testing and monitoring
Implement Delta Lake features such as time travel, vacuum operations, and Z-ordering
Develop and maintain complex ETL/ELT pipelines processing large-scale datasets daily
Design and implement CI/CD workflows for data pipelines using Databricks Asset Bundles or equivalent tools
Create real-time and batch data processing solutions with Structured Streaming and Delta Live Tables
Optimize Spark jobs for cost efficiency and performance, leveraging cluster auto-scaling and resource management
Develop custom integrations with Databricks APIs and external systems
Design scalable data architectures using Unity Catalog, Delta Lake, and Apache Spark
Establish data mesh architectures with governance and lineage tracking

Benefity

30 minute screening call online with recruiter
45-60 minute technical call with an engineer
1 hour call with the team leader
Job offer

Codest Ltd.

17 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz