Vector Search

Concepts Covered

• Vector search
• Query vectors
• Nearest-neighbor retrieval
• Similarity metrics
• Candidate generation
• Exact and approximate search
• Filters
• Latency, recall, and ranking tradeoffs
• RAG retrieval paths

Definition

Vector search retrieves stored vectors that are close to a query vector under a chosen similarity or distance rule.

The request shape is:

query -> query embedding -> nearest stored vectors -> candidate results

The returned vectors usually point back to real product objects such as document chunks, products, images, code snippets, or users.

Why This Concept Exists

Embeddings create comparable vectors. A product still needs a way to find the useful neighbors among many stored vectors.

If a knowledge base has ten chunks, brute-force comparison feels trivial. If it has tens of millions of chunks and queries sit on a live answer path, the retrieval problem becomes an engineering system:

• compare fast enough
• preserve enough quality
• apply filters
• return candidates for ranking or generation

Vector search is the online retrieval mechanism that turns a query representation into candidates.

Request Lifecycle

A typical vector search path has several stages:

1. receive the user query
2. embed the query
3. apply required scope and filters
4. search for nearby vectors
5. fetch payload or metadata for candidate IDs
6. optionally rerank or combine with keyword results
7. pass selected context or results downstream

For retrieval-augmented generation, the final candidates may become context for a language model. For product search, they may become a ranked results page.

Exact And Approximate Search

Exact vector search compares against all relevant candidates and returns the mathematically closest results under the chosen rule.

That can be reasonable at small scale or for filtered subsets.

Approximate nearest-neighbor search trades some exactness for speed, memory, or throughput at larger scale. It tries to find very good nearby candidates without comparing every vector exhaustively.

This tradeoff introduces the next layer of concepts:

• ANN indexes
• recall
• index build cost
• query-time tuning
• memory pressure

Vector search is the parent concept. Indexing techniques are how production systems make it survive scale.

Tradeoffs

Vector search rarely optimizes one metric only.

Recall: did the search path retrieve the relevant neighbors?

Latency: did it do so fast enough for the user path?

Throughput: how many queries can the service handle?

Freshness: how quickly do new or changed vectors become searchable?

Filtering: can the query respect tenant, permission, language, region, or product constraints?

Payload handling: do you return just IDs, vectors, metadata, document text, or reranking inputs?

Failure Modes

Vector search can fail even when the system is healthy.

• a query embedding lands in the wrong neighborhood
• approximate search drops a crucial candidate
• a metadata filter is too strict or applied poorly
• vector similarity retrieves related chunks without answer-bearing evidence
• stale indexes hide recent documents
• large payload fetches dominate latency after search itself succeeds

The search layer should be evaluated with task questions, not only infrastructure benchmarks.

Product Examples

In a RAG assistant, vector search finds candidate chunks before prompt assembly.

In semantic product search, vector search can surface items whose descriptions match intent without exact term overlap.

In recommendation candidate generation, vector search can find nearby users or items before heavier ranking.

In code search, it can connect a natural-language task to code fragments worth inspecting.

Related Topics

• Vector Embeddings
• Semantic Space
• Vector Databases
• Retrieval In Plain English