Data Engineer

ITFS is a consulting company from Trójmiasto with a branch in Warsaw. It focuses primarily on IT specialist recruitment and creating project-based Software Houses for clients from Poland and Western Europe.

• Minimum 5 years of commercial experience in Data Engineering or Big Data
• Very good knowledge of Spark and Scala for processing large data volumes
• Advanced knowledge of Python for building and maintaining data pipelines
• Comfortable working in Linux environment
• Experience with distributed data processing systems and Big Data platforms
• Practical knowledge of Hadoop ecosystem including Hive, Oozie, and MapReduce
• Very good knowledge of SQL and working with databases
• Ability to design, create, and develop data flows
• Very good knowledge of version control systems Git and Bitbucket
• Experience with unit testing (JUnit 5, Mockito, Spark testing)
• Ability to work in an environment requiring high code quality and process automation
• English language proficiency enabling fluent communication (minimum B2+)

Nice to have:

• Knowledge of code versioning strategies and branching
• Experience working in Agile/SAFe environments
• Knowledge of Continuous Delivery in Big Data environments
• Additional experience with Java combined with Spark and Python

• Build and develop distributed Big Data pipelines for processing large volumes of structured and unstructured data in near real-time
• Develop and maintain data processing workflows using Spark and Scala
• Create and develop data flows supporting decision-making processes and data products
• Maintain and develop solutions based on Hadoop and Big Data platforms
• Automate processes and ensure their repeatability and reliability
• Collaborate with technical teams to deliver and maintain data engineering solutions