CodiLime

Data Engineer

16.5k - 28k PLN16 500 - 28 000 PLN/ mies./ mies.B2BB2B (netto)

SeniorZdalnie·WarszawaFull-time·B2B

#429474·Dodano miesiąc temu·43

Źródło: nofluffjobs.com

Aplikuj teraz

Tech Stack / Keywords

PythonData structuresAPIpytest ETL Snowflake Apache Airflow SQLData modelingDocker KubernetesPublic cloudVersion control systemAICommunication skillsApache SparkWeb ApiFastAPI Azure AWS Go Rust Scala

Firma i stanowisko

You will join the team behind a large-scale, centralized data platform built for a global consulting organization. The platform powers several internal products and supports Mergers & Acquisitions (M&A) engagements with firm-level and site-level data, including firmographics, technographics, hierarchical relationships, and business metrics such as revenue and web traffic. The goal is to maintain a unified, enterprise-grade dataset of 300M+ company records integrated from 10+ sources.

Wymagania

Strong Python experience including data structures, typing, error handling, generators/iterators, context managers, standard library
Fundamentals of software engineering: modular design, dependency management, packaging, API design
Testing discipline including pytest, fixtures, mocking, and testable code design
Hands-on experience building and operating ETL/ELT pipelines in production
Strong experience with Snowflake and dbt
Experience with Apache Airflow or similar orchestration tools
Solid working knowledge of SQL and data modeling for schema design and queries
Experience with Docker, Kubernetes, and CI/CD practices
Debugging and profiling skills for Python performance, concurrency, and memory
Experience with at least one public cloud: AWS or Azure
Experience with version control systems (Git)
Ability to explain technical trade-offs to technical and business audiences
Experience using AI coding assistants such as Claude Code, Cursor, or similar daily
Strong communication skills and good command of English (minimum C1 level)

Nice to have:

Experience with Apache Spark, ideally on Databricks
Experience with Pydantic or similar schema-validation libraries
Experience with Python web/API frameworks (FastAPI, Flask)
Asynchronous Python, multiprocessing, or other concurrency patterns
Experience with Azure AI Search or AWS OpenSearch
Knowledge of a second programming language: Go, Rust, Scala, or TypeScript
Familiarity with LLMs, Azure OpenAI, or agentic AI systems

Obowiązki

Engineering & delivery (~70%):

Design, build, and maintain batch and streaming data pipelines in Python, including orchestration, scheduling, and monitoring with Airflow
Write reusable, well-typed Python libraries and internal packages
Build Python services and APIs using FastAPI that expose data to downstream applications and integrate with third-party/internal APIs
Develop data transformation logic in SQL and dbt on Snowflake, build data models for fast querying
Write unit, integration, and data-contract tests and maintain automated CI coverage
Profile and optimize Python code and data processing jobs for performance and cost
Deploy and operate code in the cloud using containers, infrastructure as code, and CI/CD systems
Enforce access controls, secrets handling, and data protections throughout the data lifecycle
Use AI coding assistants effectively while validating generated output and help build automated quality gates

Continuous improvement (~30%):

Find and fix efficiency, reliability, cost, and correctness issues in pipelines with refactoring
Replace scripts and notebooks with tested, packaged, scheduled code
Implement data quality checks, validation, and monitoring
Create matching logic to deduplicate and connect entities across multiple data sources
Document data processes, system architecture, and maintain project documentation

Benefity

Flexible working hours and work approach: fully remote, office, or hybrid
Professional growth supported by internal training sessions and training budget
Solid onboarding with hands-on approach
Great professional atmosphere
Ability to change projects
Sport subscription
Training budget
Private healthcare
Flat structure
Small teams
International projects
Masterclass training
Free coffee
Bike parking
Playroom
Free beverages
Free lunch
In-house trainings
Modern office
No dress code

Elastyczne godziny

Karta sportowa

Dofinansowanie szkoleń

Opieka zdrowotna

Spotkania integracyjne

Szkolenia wewnętrzne

Parking dla rowerów

Napoje w biurze

Darmowe przekąski

CodiLime

47 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz