Data Engineer
Brak informacji o wynagrodzeniu
MidFull-time·Umowa o pracę
#338917·Dodano dziś·0
Źródło: theprotocol.itTech Stack / Keywords
PythonApache SparkSQLAzure SQLKubernetes
Firma i stanowisko
Jeppesen ForeFlight is a leading provider of innovative aviation software solutions, serving the Commercial, Business, Military, and General Aviation sectors globally. Combining Jeppesen’s 90-year legacy of accurate aeronautical data with ForeFlight’s expertise in cutting-edge aviation technology, the company delivers an integrated suite of tools designed to enhance safety, improve operational efficiency, and sharpen decision-making.
Wymagania
- Minimum 3 years of experience using Python for data preparation and pipeline development
- Minimum 2 years of experience with Apache Spark / PySpark and Delta Lake for distributed data processing at scale
- Minimum 2 years of experience modeling and architecting SQL-based analytical data ecosystems (data lakehouse or data warehouse)
- Minimum 1 year of experience working in Azure cloud (or comparable cloud environment) developing batch data pipelines
- Minimum 1 year of experience with Databricks (or comparable managed Spark platform) — including jobs, workflows, and Unity Catalog
- Minimum 1 year of experience supporting data science and analytics projects and/or infrastructure
- Proficiency in Python and the PySpark DataFrame API
- Experience with Git version control and collaborative development workflows
- Must be legally able to work in Poland (or meet local employment eligibility requirements)
Nice to have:
- Technical degree/diploma in a related field of study (or equivalent experience)
- Experience developing and deploying APIs and microservices (e.g., Kubernetes)
- Strong foundation in data modeling methods including conceptual, logical, and physical data models
- Experience with development planning and version control tools (e.g., Git, Jira)
- Experience with containerization (Docker, Buildah, Podman, etc.) and CI/CD
- Experience in data quality engineering (testing, validation frameworks, anomaly detection) is an asset
- Experience with orchestration/pipeline platforms (e.g., Airflow, Databricks) is a plus
Obowiązki
- Build and maintain data lakehouse infrastructure on Databricks (Azure) supporting aviation analytics, reporting, and machine learning
- Design, build, and operate PySpark/Delta Lake ETL pipelines following a medallion architecture and data lakehouse architecture
- Propose and implement data engineering solutions for different processing strategies
- Monitor and maintain data quality, integrity, and timeliness
- Design and optimize scalable, high-performance ETL pipelines
- Implement and support CI/CD for data pipelines using Azure Pipelines, GitHub Actions and Databricks Asset Bundles
- Implement access-control and security mechanisms using Unity Catalog, OAuth/OIDC service principals, and Databricks secrets management
- Support ML integration using MLflow on Databricks for model registry, and feature pipelines
- Translate aviation business needs into technical requirements and data products
- Contribute to technical documentation
- Collaborate with data analysts, data scientists, aviation SMEs, and organizational leaders to identify opportunities for process improvement and productization
Oferta
- Private medical insurance
- Life insurance
- PPK pension plan with additional 2.5% voluntary Employer contribution
- Access to Multikafeteria platform with a wide selection of products and services, including MultiSport card
- Employee Assistance Program
- Employee referral program
- Flight rewards and flight stipend
Opieka zdrowotna
Ubezpieczenie
Karta sportowa
Płatny urlop
Bonusy
Inne informacje
Must be legally able to work in Poland (or meet local employment eligibility requirements)
JUNIPER POLAND
Pracodawca