Senior Data Engineer AI Focus

Virtusa helps its Clients by becoming a true extension of their software and data development capabilities through comprehensive and self-governing teams. This role involves participation in an up-and-coming project for a client from the high-tech area, with consultants working remotely from Poland.

• Proficiency in Python and SQL.
• Experience with Java or Scala is a plus.
• Hands-on experience with Big Data tools like Apache Spark, Flink, or Kafka.
• Familiarity with AI/ML projects, including handling embeddings, managing datasets for LLM fine-tuning, or using tools like LangChain or LlamaIndex.
• Comfortable with Infrastructure as Code tools like Terraform or container orchestration using Docker/Kubernetes.
• Advanced experience with BigQuery optimization and partitioning.
• Professional experience with managed services like Dataflow/Dataproc (Apache Beam or Spark).
• Understanding of REST APIs and microservices architecture.

• Design and scale data architectures for Machine Learning lifecycles, including feature stores, vector databases, and model training pipelines.
• Architect, build, and maintain robust ETL/ELT pipelines that handle structured and unstructured data with low latency and high reliability.
• Implement automated testing and monitoring for data integrity ensuring high-quality input for AI models.
• Build real-time event-driven architectures.
• Work with REST APIs and microservices architecture.