Big Data Engineer

Developer hired for this position will be working on implementing multiple data requirements coming from the Group Financial Crime Prevention unit. Data requirements spans from developing streaming application that connect to multiple data sources both internally or externally, but also to enable advanced analytics pipeline that produces clean, transformed data set that are being utilized to feed multiple products within the Financial Crime sector.

• Collaborative and achieve expectations through communication and teamwork in Agile setup (SAFe Framework)
• Curious, responsive and can understand the needs of others to ensure delivery of the desired results
• Work qualitatively and strive to always do things better
• High level of self-motivation
• 7+ years of experience in Spark & Scala
• 5+ years of development experience with Python
• 3+ years experience with Robot Framework
• 3+ years experience with CI/CD tools such as Jenkins
• Experience with distributed data processing engines like Spark
• Experience with Hadoop and (Hive, Oozie, map reduce, etc)
• Strong SQL skills with previous experience in creating data flows
• BitBucket and GIT expert
• AWS knowledge about architectural components (Lambda, step functions, state machine, etc)
• Unit testing experience (Junit 5, Mockito, Spark testing)
• Code versioning strategy & Branching strategy
• Advanced English skills
• Strong working experience with Linux and bash
• Experience using Jenkins or Bamboo, Jira, Bitbucket

Nice to have:

• Experience with containerization using Docker
• Knowledge of streaming and queues technologies e.g. KAFKA, IBM MQ
• Working knowledge of DevOps

• Building distributed and highly parallelized BigData processing pipeline which process massive amount of data (both structured and unstructured) in near real-time
• Leverage Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics
• Requirement analysis and questioning requirements when they are not clear
• Work closely with DevOps, QA and Product Management teams in a Continuous Delivery environment
• Test case and test data preparation
• Test automation scripts implementation in Robot Framework and Python
• Test planning and test execution (test results analysis and defect reporting)