Data Engineer z Hadoopem

• Experience with Pyspark or Scala development and design
• Experience using scheduling tools such as Airflow
• Knowledge of Hadoop ecosystem including Spark, Hive, YARN, and ETL frameworks
• Strong SQL and RESTful services knowledge
• Experience working on Unix or Linux platforms
• Hands-on experience building data pipelines using Hadoop components
• Experience with Git, GitHub, Jenkins, Ansible, and JIRA
• Understanding of big data modelling using relational and non-relational techniques
• Experience debugging code and communicating findings to development teams

Nice to have:

• Experience with Elasticsearch
• Experience developing Java APIs
• Experience in data ingestion processes
• Understanding of cloud design patterns
• Exposure to DevOps and Agile methodologies such as Scrum and Kanban
• Experience with Spark streaming
• Experience with Apache Airflow in production
• Experience with Hadoop ecosystem in enterprise environments
• Knowledge of Python backend services
• Experience with Scala for high performance systems
• Experience in data integration and ETL processes
• Knowledge of PL/SQL
• Experience with Linux and Unix system operations

• Develop and design using Pyspark or Scala
• Use scheduling tools such as Airflow
• Build data pipelines with Hadoop components
• Debug code and communicate findings to development teams
• Work on Unix or Linux platforms
• Utilize tools including Git, GitHub, Jenkins, Ansible, and JIRA