Big Data Engineer (Scala, Spark)

Brak informacji o wynagrodzeniu

MidOn-siteFull-time

#265587·Dodano 27.01.2025·27

Źródło: LinkGroup

Aplikuj teraz

Tech Stack / Keywords

Big Data Scala Spark HiveNoSQLAWS Azure GCP

Firma i stanowisko

Link Group specializes in building tech teams for Fortune 500 companies and startups, connecting professionals with career opportunities. The project involves building a high-performance data platform for the finance/stock exchange industry, focusing on real-time analytics and reporting with scalability and efficiency.

Wymagania

At least 3+ years of experience in big data engineering.
Proficiency in Scala and experience with Apache Spark.
Strong understanding of distributed data processing and frameworks like Hadoop.
Experience with message brokers like Kafka.
Hands-on experience with SQL/NoSQL databases.
Familiarity with version control tools like Git.
Solid understanding of data architecture and ETL processes.
Good command of English.

Nice to have:

Experience with cloud platforms such as AWS, Azure, or GCP.
Knowledge of containerization tools like Docker and orchestration with Kubernetes.
Familiarity with CI/CD pipelines for big data workflows.
Experience in optimizing Spark jobs for large-scale data processing.
Academic background in Computer Science, Data Engineering, or a related field.

Obowiązki

Design, implement, and optimize big data processing pipelines using Scala and Spark.
Work with massive datasets, ensuring scalability and performance.
Collaborate with data scientists and analysts to develop data solutions.
Monitor and troubleshoot data workflows to ensure reliability and efficiency.
Integrate with data sources and ensure proper data ingestion and transformation.
Research emerging big data technologies to improve the architecture.

Benefity

Tailored opportunities to match professional interests and goals.
A dynamic and collaborative work environment.
Access to diverse projects for global clients.
Competitive compensation aligned with expertise.
Ongoing opportunities for professional growth and development in big data.

Link Group

550 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz