Data Flow Manager

Chabre IT Services is a global professional IT services provider specializing in tailor-made solutions, smart outsourcing, try&hire, and success fee services.

• Proven experience in building, maintaining, and monitoring complex data pipelines and data flow architectures.
• Hands-on proficiency with Cloudera DataFlow and Apache NiFi.
• Solid knowledge of real-time streaming technologies, particularly Apache Kafka.
• Practical experience with Change Data Capture mechanisms including Debezium and SQL CDC.
• Familiarity with integration standards and protocols such as REST API and JDBC.
• Experience managing data schemas (Avro) and working with metadata tracking tools like Apache Atlas.
• Understanding of data security and governance principles, with ability to configure policies using Apache Ranger.
• Strong analytical skills with experience troubleshooting performance bottlenecks and automating operational tasks.
• Excellent communication skills for collaboration and producing architectural documentation.

• Design, implement, evaluate, and support sophisticated data processing pipelines within Cloudera DataFlow (Apache NiFi), managing data ingestion, transformation, enrichment, and routing.
• Develop and fine-tune Change Data Capture (CDC) architectures for real-time or near-real-time data processing using NiFi, Kafka, Debezium, or SQL CDC connectors.
• Establish integrations between internal architectures and external data sources using protocols such as REST API, JDBC, and Kafka.
• Oversee data structures and lifecycles by maintaining Avro schemas and managing metadata and data lineage through Apache Atlas.
• Ensure data security and governance by configuring policies and utilizing Apache Ranger.
• Monitor pipeline performance, troubleshoot stability issues, and automate data flow processes.
• Collaborate with business stakeholders, architects, and engineers to define system architecture, draft flow documentation, and author SOPs and runbooks.
• Support system enhancements by participating in upgrades of platforms like CDP, NiFi, and Kafka, and evaluating new data connectors.