Senior Data Engineer – AWS & Databricks (k/m)

Lumicode Sp. z o.o. is part of the Pentacomp Group, which for nearly 30 years has been delivering IT solutions and technology services for major enterprises and public sector institutions. They execute projects supporting the digital transformation of clients, combining experience with modern technologies.

• Proficient experience with AWS cloud services used for data processing and storage
• Practical experience building and maintaining data engineering processes with Databricks
• Very good knowledge and ability to work with large datasets using SQL
• Experience designing scalable pipelines and data models
• Building ETL/ELT processes and working with data warehouses/data platforms
• Optimizing processes for performance, cost, and reliability
• Working with data from various sources and transforming it for analytics use
• Following best practices for code quality, testing, and version control
• Independent problem solving and decision making on task implementation
• English communication skills at minimum B2+ level

Nice to have:

• Python experience in automating data engineering processes
• Experience with orchestration tools like Airflow
• Knowledge of AWS S3 and Athena
• Experience building CI/CD pipelines, e.g., with GitHub Actions
• Experience in AdTech / Programmatic industry

• Design and develop scalable data pipelines in the AWS and Databricks environment
• Build data processing workflows from raw data layers to analytics and product-used layers
• Design and develop data models and ETL/ELT processes
• Work with large datasets optimizing performance and costs
• Develop and maintain data processing in Databricks
• Integrate data from various sources
• Monitor pipelines and diagnose data quality and performance issues
• Ensure code quality, testability, and apply software engineering best practices
• Collaborate with Data Engineers, Data Product Managers, and analytics teams
• Support development of data engineering architecture and standards

The application process involves personal data processing consent under GDPR by Lumicode Sp. z o.o., part of the Pentacomp group, with data processed for recruitment and future recruitment purposes. Data is stored as per legal requirements, and rights to access, correct, delete data, and to withdraw consent are outlined. No automated decisions or profiling are made based on personal data.