Data Engineer (Data Center)

Our client is a leading global data center provider delivering hyperscale and edge infrastructure solutions across the Americas, EMEA, and Asia-Pacific. With 80+ data centers in 20+ countries, they partner with industry leaders such as Google, Oracle, NVIDIA, and Microsoft Azure to power the world’s digital infrastructure.

• 7+ years of experience in data or software engineering, including production experience with streaming systems.
• Ability to work independently in ambiguous and fast-changing environments.
• Deep hands-on experience with Apache Kafka and Apache Flink or equivalent stream-processing framework, including Java.
• Strong Python and SQL skills across transactional and analytical databases.
• Experience with Apache Iceberg, Delta Lake, or Hudi; Parquet, Avro, schema registries, Athena, or Spark SQL.
• Production experience with ClickHouse, Druid, Pinot, or similar analytical databases, as well as SQL Server or PostgreSQL.
• Experience with Kubernetes, Amazon EKS, Helm, Argo CD, and GitOps.
• Experience with Terraform and GitHub Actions.
• Strong knowledge of AWS services, including MSK, EMR Serverless, MWAA, S3, Athena, and IAM.
• Knowledge of ML fundamentals including feature engineering, model training and evaluation, and ML data requirements.
• Familiarity with LLMs, prompt-based workflows, embeddings, vector search, anomaly detection, and forecasting.
• Experience with observability, alerting, troubleshooting, and incident response.
• Ability to communicate technical decisions clearly to technical and non-technical stakeholders.

Nice to have:

• Working knowledge of Azure Event Hubs, Data Factory, and Synapse.
• Experience with OT or industrial telemetry including OPC UA, BMS/EPMS, or time-series sensor data.
• Experience with cloud-to-cloud or on-premises-to-cloud migrations.
• Experience with data center or other critical-infrastructure operations.
• Familiarity with data quality frameworks, data contracts, and metadata management.

• Design, build, and operate real-time streaming pipelines using Apache Kafka (Amazon MSK) and Apache Flink for high-throughput sensor and telemetry data.
• Manage streaming data contracts and schema evolution using Avro and a schema registry.
• Build and maintain analytical serving layers using columnar OLAP databases (ClickHouse or similar) and REST APIs that expose data to dashboards, applications, and downstream teams.
• Develop and operate batch and scheduled workflows with Apache Airflow (Amazon MWAA).
• Build and run the data lakehouse: Apache Iceberg on Amazon S3, PySpark on EMR Serverless, and Amazon Athena.
• Deploy and operate containerized workloads on Kubernetes (Amazon EKS) using Argo CD, Helm, and GitOps practices.
• Manage infrastructure as code with Terraform and CI/CD pipelines with GitHub Actions.
• Monitor and maintain production data platforms using Prometheus and Grafana, and participate in incident response.
• Contribute to cloud migration work, porting data pipelines from legacy platforms including Azure to AWS.
• Write high-quality, production-grade code in Java, Python, and SQL with automated testing.
• Keep component documentation and runbooks current with implementation.

Minimum overlap with EST time zone - 4 -5 hours.