Vector Search in Production with Elasticsearch

Swapping keyword search for vector search on a large, messy product catalog sounds simple until you actually try it. This is what moved relevance by roughly 35–45% on a real deployment — not a single trick, but a handful of decisions that compound.

Picking an embedding model

General-purpose sentence embedding models work reasonably out of the box, but fine-tuning (or at least carefully prompt-templating the text you embed) on your own product titles and descriptions closes most of the remaining gap. I embed a normalized string — brand, name, key attributes, category — rather than raw scraped titles, since raw titles are inconsistent across retailers.

Indexing in Elasticsearch

Elasticsearch's dense_vector field type with HNSW indexing handles approximate nearest-neighbor search at catalog scale without a separate vector database. A few settings mattered more than expected:

• ef_construction / m tuning — trading index build time for recall; worth raising past the defaults for a catalog that doesn't change every second.
• Quantization — reducing vector precision cut memory footprint significantly with negligible relevance loss.
• Field boosting — combining the vector score with boosted lexical fields (brand, exact SKU) kept exact matches from getting buried under "semantically close" results.

Evaluating relevance, not just recall

It's easy to ship vector search and assume it's better because it "understands" queries. I built a small labeled evaluation set of real user queries with graded relevance judgments and tracked NDCG before and after each change — that's where the 35–45% number actually came from, not intuition. Anything that can't be measured this way doesn't ship.