Data Engineer

CodiLime is a software and network engineering industry expert and the first-choice service partner for top global networking hardware providers, software providers, and telecoms. The company has over 250 employees and has been operating since 2011. It serves both tech startups and large players in various industries across multiple geographic locations including the US, Japan, Israel, and Europe.

• Strong experience with Python including data structures, typing, error handling, generators/iterators, context managers, and standard library
• Software engineering fundamentals: modular design, dependency management, packaging, API design
• Testing discipline using pytest, fixtures, mocking
• Hands-on experience building and operating production ETL/ELT pipelines
• Strong experience with Snowflake and dbt
• Experience with Apache Airflow or similar orchestration tools
• Solid working knowledge of SQL and data modeling
• Experience with Docker, Kubernetes, and CI/CD practices
• Debugging and profiling skills in Python
• Experience with at least one public cloud (AWS or Azure)
• Experience with version control systems (Git)
• Ability to explain technical trade-offs to technical and business audiences
• Daily use of AI coding assistants such as Claude Code or Cursor
• Strong communication skills and good knowledge of English (minimum C1 level)

Nice to have:

• Experience with Apache Spark, ideally on Databricks
• Experience with Pydantic or similar schema-validation tools
• Web/API frameworks in FastAPI, Flask
• Async Python, multiprocessing, or concurrency patterns
• Experience with Azure AI Search or AWS OpenSearch
• Knowledge of a second language: Go, Rust, Scala, or TypeScript
• Familiarity with LLMs, Azure OpenAI, or agentic AI systems

Engineering & delivery (~70%):

• Design, build, and maintain batch and streaming data pipelines in Python, including orchestration with Airflow
• Write reusable, well-typed Python libraries and internal packages
• Build Python services and APIs using FastAPI and integrate with third-party/internal APIs
• Develop data transformation logic in SQL and dbt on Snowflake
• Write unit, integration, and data-contract tests, and maintain CI coverage
• Profile and optimize Python code and data processing jobs
• Deploy and operate code in the cloud using containers, infrastructure as code, and CI/CD
• Enforce access controls, secrets handling, and sensitive data protections
• Use AI coding assistants effectively while validating generated output

Continuous improvement (~30%):

• Find and fix efficiency, reliability, cost, and correctness issues in pipelines
• Replace scripts and notebooks with tested, packaged, scheduled code
• Implement data quality checks, validation, and monitoring
• Create matching logic to deduplicate and connect entities across data sources
• Document data processes and system architecture