OpenSearch Engineer / AWS & Data Pipelines

Ruby Labs is a leading tech company that creates and operates innovative consumer products across health, education, and entertainment industries.

• Strong production experience with OpenSearch or Elasticsearch.
• Hands-on experience with AWS OpenSearch Service.
• Experience building data pipelines between PostgreSQL and OpenSearch.
• Strong understanding of index and shard design, mappings and analyzers, OpenSearch Query DSL, aggregations, sorting, filtering, pagination, bulk indexing, cluster performance, data consistency, retries, idempotency, and replay.
• Experience with large datasets and high indexing or query volumes.
• Experience performing backfills and reindexing safely.
• Strong troubleshooting and production-operations skills.

Nice to have:

• Experience with AWS DMS or OpenSearch Ingestion.
• Experience with Lambda, SQS, S3, ECS, EKS, or Terraform.
• Experience building custom ingestion services in Go, Java, or Python.
• Experience in payments, billing, fintech, or subscription systems.
• Experience with multi-tenant or high-volume search platforms.
• Familiarity with ClickHouse, Tinybird, or similar analytical systems.
• AWS certifications.

OpenSearch:

• Own and operate AWS OpenSearch Service in production.
• Design indices, mappings, analyzers, templates, aliases, shards, and retention policies.
• Optimize search queries, aggregations, filtering, pagination, and bulk indexing.
• Investigate slow queries, indexing bottlenecks, hot shards, memory pressure, and rejected requests.
• Plan safe upgrades, migrations, reindexing, and capacity scaling.

PostgreSQL-to-OpenSearch pipelines:

• Build and maintain reliable pipelines that synchronize PostgreSQL data with OpenSearch.
• Support inserts, updates, deletes, backfills, and full reindexing.
• Use AWS-native services such as AWS DMS, OpenSearch Ingestion, Lambda, SQS, S3, ECS, or EKS where appropriate.
• Implement retries, idempotency, deduplication, replay, dead-letter handling, and failure recovery.
• Monitor ingestion lag, throughput, failed records, and data freshness.
• Build reconciliation processes to detect and repair missing, stale, or inconsistent documents.
• Ensure ingestion does not negatively affect the transactional PostgreSQL workload.

Reliability and operations:

• Monitor cluster health, search latency, indexing latency, storage, CPU, JVM pressure, and shard distribution.
• Build CloudWatch dashboards, alerts, and operational runbooks.
• Design snapshot, restore, backup, and disaster-recovery procedures.
• Participate in production incident response and root-cause analysis.
• Implement secure access using IAM, role-based access control, encryption, and least-privilege principles.

Collaboration:

• Work closely with backend, database, infrastructure, and product teams.
• Help design efficient OpenSearch document models for payments, customers, subscriptions, and transactions.
• Define standards for searchable fields, schema changes, and index versioning.
• Document architecture, data flows, and recovery procedures.

Applicants must be located within approximately ± 4 hours of Central European Time (CET) zone for optimal collaboration and communication.