Definition
Multi-vector retrieval represents one logical document or item with multiple vectors instead of one vector.
The plain-English version:
one document can have many searchable representations
Those vectors may represent chunks, passages, sections, summaries, fields, or token-level signals.
Why This Concept Exists
A single vector can be too compressed.
Imagine a long product document covering:
- pricing
- cancellation
- security
- admin roles
- integrations
One embedding for the whole document may blur those topics together.
Multi-vector retrieval exists because many documents contain multiple meanings, and different user queries may match different parts.
The Beginner Mental Model
A beginner may think:
Each document gets one embedding.
That is the simplest design, not the only design.
A better mental model:
logical item -> many vectors -> one returned document or item
The system may search across the smaller representations, then return or rank the parent item.
Common Shapes
Multi-vector retrieval can appear in several forms.
Chunk vectors: each document chunk gets its own vector.
Summary vectors: the same document has summaries optimized for retrieval.
Field vectors: title, body, tags, and metadata each get separate representations.
Token-level vectors: models like late-interaction retrievers can keep finer-grained token representations.
Question vectors: generated likely questions point back to the source document.
The shared idea is that retrieval should not force one vector to carry every useful matching signal.
A Concrete Example
A security policy document has one section about passwords and another about audit logs.
A user asks:
How long do audit logs need to be retained?
If the whole document has one vector, password content may dilute the audit-log signal.
With multi-vector retrieval, the audit-log section can have its own representation and match the query more directly.
The result can still return the parent policy document or section.
Late Interaction
Some retrieval models encode a document into multiple token-level or passage-level vectors and perform a late interaction with the query representation.
The intuition:
do not compress all document meaning into one point too early
This can improve retrieval quality, but it usually costs more storage and more complex scoring than single-vector retrieval.
Tradeoffs
Multi-vector retrieval improves representation flexibility, but it adds cost.
Engineers must manage:
- more vectors to store
- more index entries
- parent-child mapping
- deduplication
- score aggregation
- update and deletion paths
- higher retrieval complexity
The question is not "can we create more vectors?" It is "do the extra vectors improve retrieval quality enough to justify the cost?"
Failure Modes
Multi-vector retrieval can fail when:
- many vectors from one document dominate results
- score aggregation hides the best evidence
- duplicate parent documents crowd the context window
- update jobs leave old child vectors behind
- permission checks happen at the wrong level
- storage cost grows faster than retrieval quality
The system needs careful grouping and evaluation.
Common Confusions
Multi-vector retrieval is not the same as hybrid search.
Hybrid search combines different retrieval signals. Multi-vector retrieval uses multiple vector representations for one item.
Multi-vector retrieval is not automatically better.
It can improve recall and matching detail, but it adds storage and ranking complexity.
A retrieved vector is not always the final context.
The vector may point to a parent item, section, or payload that still needs selection.
Token-level retrieval is not required for every RAG system.
Many products do well with chunk vectors plus reranking.
What This Does Not Mean
Multi-vector retrieval does not remove context engineering.
The system still has to decide which parent context enters the prompt and how much evidence is enough.