Elasticsearch Principal Engineer

We are looking for a highly skilled Elasticsearch Expert who can architect, optimize, and evolve our entire search ecosystem. This person will own how search works across millions of products, ensure high-performance indexing and querying, design relevance algorithms, and make sure we utilize Elasticsearch to its maximum capabilities.

You will work closely with Engineering, Data, DevOps and Product teams to ensure our search infrastructure is scalable, resilient, cost-efficient, and delivers the best shopping experience.

Key Responsibilities

1️ Search Architecture Ownership

Design, build, and optimize Elasticsearch clusters for high availability, low latency, and large-scale indexing.

Define index mappings, templates, analyzers, normalizers, tokenizers and custom pipelines suitable for multilingual, e-commerce data.

Architect multi-index strategies (aliases, versioned indices, reindexing flows, rollover, hot-warm tiers).

Implement proper sharding, replication, and routing strategies.

2️ Search Relevance & Ranking

Build relevance ranking systems: boosting, scoring, weighted factors, script scoring.

Implement personalization-aware scoring (behavioral signals, trending, popularity).

Enhance catalogue search for precision, recall, typo handling, and semantic relevance.

Design category-level dynamic ranking and scoring logic.

3️ Advanced Query Engineering

Build dynamic filtering systems (sizes, colors, categories, brands, attributes).

Create complex real-time aggregations for availability, pricing, and insights.

Implement vector search for embeddings (OpenAI or AWS-based semantic search).

Optimize keyword search, fuzzy matching, synonyms, stemming rules, stop-words.

4️ Performance & Scalability

Optimize indexing throughput and speed for millions of documents.

Solve cluster bottlenecks: heap usage, GC tuning, threadpools, cache layers.

Manage ILM (Index Lifecycle Management) for hot/warm/cold architecture.

Design highly available clusters with fault tolerance.

5️ Observability & Maintenance

Implement rich monitoring dashboards (Grafana, Kibana, CloudWatch).

Setup alerts for cluster health, indexing failures, slow queries, JVM pressure.

Perform periodic audits to ensure the cluster remains efficient and stable.

6️ Data Pipeline & Integration

Work with backend teams to design efficient indexing pipelines (queue-based or batch).

Ensure clean, structured, normalized data models for indexing.

Build fallback strategies for partial failures and safe reindexing rollout.

7️ Cost Optimization & Best Practices

Continuously monitor cluster costs and implement optimization:

Hot vs Warm storage

Shard count reduction

Compression techniques

Scaling policies

Recommend cluster upgrades or migration improvements.

Skills & Experience Needed

Must-Have Technical Skills

Strong expertise in:

Mapping design

Custom analyzers

Aggregations

Relevance tuning

Cluster scaling & performance optimization

Index lifecycle management (ILM)

Deep understanding of:

Inverted indices

Doc_values

Fielddata

HNSW vector search

Query DSL & complex scoring logic

Hands-on experience with:

ECS / Kubernetes deployments

AWS managed Elasticsearch / OpenSearch

Node.js or Python-based indexing pipelines

Queues (Kafka/SQS) for ingest

Bonus Skills

Experience building search for e-commerce platforms.

Experience with semantic search, embeddings, LLM integrations.

Experience with A/B testing of ranking algorithms.

Strong data-modelling experience for multi-attribute product catalogs.

Familiarity with log search, fraud signals search, personalization engines.

Job Types: Full-time, Permanent, Contract
Contract length: 12 months