Retrieval begins by deciding what a query and a document should look like to a search system. The same sentence can be represented as lexical terms, a sparse learned vector, one dense embedding, many token-level vectors or a hybrid combination of several signals.
Retrieval representation is the mechanism layer between source text and the evidence set returned to a reader or language model.
This article extends the eduKateSingapore Representation and Tokenisation series beyond search tokenisation into modern retrieval geometry.
The Retrieval Representation Route
QUERY + DOCUMENTS → LEXICAL / SPARSE / DENSE / TOKEN-LEVEL REPRESENTATIONS → INDEX → CANDIDATE RETRIEVAL → SCORE FUSION OR RERANKING → EVIDENCE SET → CONTEXT SUFFICIENCY TEST → ANSWER / SEARCH RESULT
1. Search Representation Determines What Can Match
If the system represents a query only through exact terms, paraphrases may be missed. If it represents only semantic similarity, exact identifiers can be diluted.
Retrieval quality begins with the chosen representation family.
2. Lexical Retrieval Uses Sparse Term Signals
Traditional systems such as BM25 represent documents and queries through term occurrence statistics over a large vocabulary.
Most vocabulary dimensions are zero for any one document, making the representation sparse.
3. Sparse Lexical Signals Preserve Exact Words
Rare names, error codes, product IDs and unusual technical terms can match strongly when they appear exactly.
This is a major strength of sparse retrieval.
4. Exact Words Can Miss Paraphrase
A user asks “How do I cancel?” while the document says “terminate the subscription”.
Lexical mismatch can hide semantically relevant evidence.
5. Stemming and Expansion Try to Bridge Lexical Gaps
Classical systems can stem related word forms or expand queries with related terms.
These techniques enrich sparse representation without abandoning explicit vocabulary dimensions.
6. Learned Sparse Retrieval Makes Expansion Contextual
SPLADE v2 learns sparse lexical representations whose non-zero vocabulary dimensions can include useful expansion terms not literally present in the source.
The representation remains indexable through sparse inverted-index machinery while gaining learned semantic expansion.
7. Learned Sparse Vectors Are Still Interpretable at Vocabulary Level
Because dimensions correspond to vocabulary terms, engineers can inspect which terms receive weight.
This offers more lexical transparency than one dense vector.
8. Dense Retrieval Maps Text Into Continuous Embeddings
A dense retriever encodes a query and document into compact numerical vectors where similarity approximates semantic relevance.
The match no longer requires shared surface words.
9. Dense Passage Retrieval Popularised Dual-Encoder Search
Dense Passage Retrieval showed that separately encoded question and passage vectors could outperform strong lexical baselines on several open-domain QA benchmarks.
The architectural benefit is indexability: document embeddings can be precomputed and searched efficiently.
10. One Dense Vector Is Strong Compression
A passage containing hundreds of tokens can become one embedding.
This improves search efficiency and inevitably loses exact token-level detail.
11. Dense Similarity Is Not Exact Identity
Two semantically related passages can sit close in vector space while differing on names, dates or negation.
Similarity should not be mistaken for factual equivalence.
12. Embedding Models Define the Geometry of Search
Changing the embedding model changes which passages count as near neighbours.
Vector indexes are therefore tied to the representation model that produced them.
13. Embedding Migration Requires Re-Indexing
Old document vectors and new query vectors should not be compared unless they share a compatible embedding space.
Retrieval representation has versioning requirements just like tokenizers.
14. Dense Indexes Need Approximate Nearest-Neighbour Search
Large vector collections are commonly searched through ANN structures rather than exhaustive comparison.
The index introduces another recall-versus-speed trade-off above the embedding itself.
15. Representation Recall and Index Recall Are Separate
A relevant passage can be far away because the embedding is weak, or it can be near and still missed by the approximate index.
Diagnose semantic representation and search infrastructure separately.
16. Late Interaction Preserves More Token-Level Detail
ColBERT encodes query and document token representations separately and performs a lightweight late interaction at search time.
This keeps more fine-grained matching capacity than compressing each passage into one vector.
17. Late Interaction Trades Storage for Matching Resolution
Storing many token vectors per document is more expensive than storing one passage vector.
The gain is richer alignment between specific query terms and specific document regions.
18. Retrieval Granularity Matters
Documents can be indexed as whole pages, paragraphs, sentences or overlapping chunks.
The representation of the item determines what context arrives together when it is retrieved.
19. Small Chunks Improve Precision and Lose Context
A sentence-sized chunk can match one fact cleanly and omit the heading or exception required to interpret it.
Fine retrieval units need contextual metadata or neighbouring evidence.
20. Large Chunks Preserve Context and Dilute Similarity
A long section can contain the answer plus several unrelated topics.
Its embedding may become a broad average rather than a precise query match.
21. Hybrid Retrieval Combines Sparse and Dense Signals
Sparse retrieval preserves exact lexical evidence. Dense retrieval captures semantic paraphrase.
Hybrid systems combine both because their errors differ.
22. Hybrid Search Needs Score Fusion
BM25 scores and embedding similarities live on different numerical scales.
Fusion can normalise scores, learn weights or combine rank positions rather than raw scores.
23. Reciprocal Rank Fusion Combines Ranked Lists Simply
RRF rewards documents that appear highly in one or more ranked lists without requiring score calibration between systems.
It is useful precisely because heterogeneous retrieval representations do not naturally share one scoring scale.
24. Rerankers Add a More Expensive Representation Stage
A first-stage retriever can fetch hundreds of candidates cheaply. A cross-encoder reranker can then jointly inspect query and candidate text with deeper interaction.
The pipeline spends high-resolution computation only on a narrowed set.
25. Retrieval Is Naturally Hierarchical
Broad sparse or dense search → finer reranking → evidence assembly is a coarse-to-fine representation process.
This mirrors Hierarchical Representation.
26. Training Positives Define What Relevance Means
A dense retriever trained on question-answer pairs learns one notion of relevance. A product-search retriever trained on purchases learns another.
Embedding geometry inherits the training objective.
27. Hard Negatives Shape Decision Boundaries
A hard negative looks superficially relevant but is not the correct evidence.
Training against hard negatives teaches the model distinctions that easy random negatives never require.
28. False Negatives Can Poison Retrieval Training
A passage labelled negative may actually contain valid alternative evidence.
Representation learning then pushes useful documents away from the query.
29. Query Rewriting Changes the Retrieval Representation Upstream
A conversational query such as “What about the second one?” can be rewritten into a self-contained search query before encoding.
Search quality can fail because the query representation was incomplete before the retriever ever saw it.
30. Metadata Filtering Is Another Retrieval Signal
Date, language, jurisdiction, document type and access permissions can restrict candidate sets before semantic scoring.
Typed filters protect distinctions embeddings may blur.
31. Currentness Needs Temporal Representation
Semantic relevance cannot tell whether a regulation, price or schedule is outdated.
Freshness needs explicit dates and update policy.
32. Authority Needs Provenance Representation
A primary regulator and a scraped summary can be semantically similar.
Source authority should travel as metadata rather than emerge accidentally from embedding distance.
33. Retrieval Quality Is Not Answer Quality
High recall can return many relevant passages while omitting one decisive premise.
Retrieved evidence should pass the Context Sufficiency test before confident generation.
34. Retrieval Needs Evaluation at Several Layers
- Representation: is relevant evidence near the query?
- Index: can the system actually retrieve those neighbours?
- Ranking: do the strongest candidates appear early?
- Diversity: are duplicates wasting slots?
- Sufficiency: does the final evidence set answer the question?
35. Retrieval Representation Audit
- What item is indexed: document, passage, sentence or token set?
- Is representation lexical, learned sparse, dense, late-interaction or hybrid?
- What tokenizer and embedding model produced it?
- What training objective defined relevance?
- What hard negatives were used?
- What exact identifiers require lexical matching?
- What paraphrase patterns require semantic matching?
- How are sparse and dense scores fused?
- What ANN or inverted index is used?
- How is index recall measured?
- What reranking stage follows retrieval?
- How are freshness, authority and permission represented?
- Are near-duplicate passages wasting top-k slots?
- Does the final set contain sufficient evidence?
36. What Students Should Remember
- Sparse retrieval preserves lexical identity.
- Dense retrieval maps queries and documents into semantic vector spaces.
- SPLADE learns sparse lexical expansion; DPR popularised dense dual-encoder passage retrieval.
- ColBERT preserves token-level matching through late interaction.
- Hybrid retrieval combines representations because their failure modes differ.
- Reranking spends deeper computation on a narrowed candidate set.
- Retrieval success must ultimately be judged by evidence sufficiency, not similarity alone.
37. The Deep Principle
A retrieval representation defines what “near” means before search begins.
The best retrieval system does not choose between words and meaning. It preserves enough lexical identity to find exact things, enough semantic geometry to find paraphrases, and enough provenance to know whether the evidence deserves to be believed.