Senior Big Data Engineer

Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. The company has 8 years of experience and leadership in enterprise AI, data, analytics, cloud & DevOps, application modernization, and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and other regions.

• Strong software development experience with Scala and deep proficiency in Apache Spark, including Dataframe/SQL API.
• Practical experience with Spark Streaming for real-time data processing.
• Good understanding of Kubernetes, specifically its use with Spark.
• Hands-on expertise with distributed storage and data lakehouse formats like Hive and Iceberg.
• Proven experience in workflow orchestration using Apache Airflow.
• Strong proficiency in SQL, with advanced knowledge of SparkSQL.
• Solid understanding of distributed computing technologies and hands-on experience with Hadoop.

Nice to have:

• Practical experience with Apache Flink for stream processing.
• Familiarity or hands-on experience utilizing Claude (Anthropic) for engineering workflows or automation.
• Experience implementing Data Lakes, Data Warehousing, or advanced analytics systems.
• Strong background in DevOps, infrastructure management, and building automated CI/CD pipelines at scale.

• Design, develop, and optimize scalable big data processing pipelines using Scala and Apache Spark.
• Implement and verify complex data jobs to ensure statistical algorithms deliver precise metrics.
• Work with infrastructure and cloud environments, leveraging Kubernetes for managing Spark on clusters.
• Orchestrate data workflows and scheduling with Apache Airflow.
• Build and maintain data streaming architectures using Spark Streaming and Apache Kafka.
• Collaborate with an international distributed team of big data engineers.
• Design and query complex data models across databases such as Oracle, Postgres, Teradata, and Cassandra.
• Integrate DevOps and CI/CD practices into the data lifecycle to manage backend infrastructure efficiently.

INFORMATION ON PERSONAL DATA PROCESSING: Grid Dynamics Poland sp. z o.o. and other entities from the Grid Dynamics Group are joint controllers of personal data. Legal bases include consent, recruitment process necessity, legal obligations, and legitimate interests. Data may be transferred outside the EEA to Group entities in various countries. Data subject rights and complaints procedures are specified. Processing duration is 3 years.