Data Flow Engineer

Brak informacji o wynagrodzeniu
SeniorFull-time
#342963·Dodano dziś·0
Źródło: InventYOU AB
Aplikuj teraz

Tech Stack / Keywords

Big DataApacheRESTKafkaSecurityPythonApache Kafka

Firma i stanowisko

inventYOU is a leading IT Consulting company founded in Sweden in 2017. Our services include professional IT services onsite and Nearshoring. Our clients range from start-ups to large enterprises. Our goal is to empower our clients to achieve their business objectives and maximize their IT investments. We strive to create innovative solutions that are tailored to our client’s needs and provide a comprehensive range of services that enable them to navigate the ever-evolving IT landscape.


Wymagania

  • Minimum 8 years of overall IT experience
  • Minimum 6 years of experience in similar positions
  • Experience working with Apache NiFi / Cloudera DataFlow environments
  • Strong experience in data integration and data flow design
  • Advanced Python programming skills (data processing, automation, integrations)
  • Strong experience building integrations using REST APIs (OAuth/JWT, error handling, rate limiting)
  • Hands-on experience with CDC pipelines and relational database integrations
  • Experience with Apache Kafka and data streaming architectures
  • Knowledge of Apache Avro and schema evolution concepts
  • Experience with Apache Iceberg (tables, schema evolution, partitioning)
  • Knowledge of data governance tools such as Apache Atlas and Apache Ranger
  • Experience working in CDP (Cloudera Data Platform) environments
  • Strong analytical and problem-solving skills
  • Fluency in English (minimum B2 level)
  • Minimum education level equivalent to Level 6 qualification
  • Valid Security Clearance is mandatory

Certifications (mandatory):

  • At least one of the following:
    • Cloudera Certified Developer for Apache NiFi (or equivalent)
    • Cloudera DataFlow (CFM) certification (or equivalent internationally recognised certification)

Obowiązki

  • Design, implement, test, and maintain complex data flows using Apache NiFi (Cloudera DataFlow)
  • Build and optimise CDC-based data pipelines (real-time / near real-time)
  • Integrate data flows with external systems using REST APIs, JDBC, Kafka, and other protocols
  • Manage data ingestion, transformation, enrichment, routing, and delivery processes
  • Configure and maintain data schemas, metadata, and lineage
  • Implement and manage security and governance policies for data flows
  • Monitor, troubleshoot, and optimise performance and reliability of data pipelines
  • Collaborate with data engineers, architects, and business stakeholders
  • Create and maintain documentation, SOPs, and runbooks
  • Participate in platform upgrades, migrations, and continuous improvement activities

Oferta

  • Work on complex and large-scale data engineering environments
  • Contribute to modern data integration and streaming architectures
  • Collaborate with experienced data engineers and architects
  • Be part of technically challenging and high-impact projects
  • Develop your expertise in big data, streaming, and data flow technologies

Inne informacje

Valid Security Clearance is mandatory

InventYOU AB

InventYOU AB

4 aktywne oferty

Zobacz wszystkie oferty
Aplikuj teraz