Embeddings, what they do and what they do not do

An embedding turns text into a list of numbers such that texts with similar meaning produce similar numbers. That property is genuinely useful and it is also narrower than it sounds.

What similarity actually captures

Embeddings are trained so that text appearing in similar contexts ends up nearby. This captures topic and register well. It does not reliably capture negation, quantity, or time.

“The Pro plan includes lead forms” and “The Pro plan does not include lead forms” are nearly identical as strings. They differ by one word that reverses the meaning entirely, and they will sit very close together in embedding space.

This is not a bug to be fixed by a better model. It is a consequence of what similarity means. If your content contains both a statement and its negation, similarity search cannot reliably tell you which one answers the question.

The practical consequences

Numbers do not embed well

“500 credits” and “5,000 credits” are close in embedding space and ten times apart in fact. Any question whose answer hinges on a specific number is at risk from pure vector retrieval. This is a strong argument for hybrid retrieval on pricing content specifically.

Recency is invisible

An embedding has no concept of when text was written. A 2024 pricing page and a 2026 pricing page embed almost identically. If both are in your index, retrieval will pick one essentially at random. The fix is at the source level: do not index superseded content.

Contradictions are undetectable at query time

If two indexed pages disagree, retrieval will surface one of them and the model will answer from it confidently. There is no mechanism that notices the disagreement. This has to be prevented at indexing time, by having a single authoritative source per fact.

What embeddings are good at

Worth being fair about the strengths, because they are the reason to use this at all:

• Paraphrase. “How much” and “what is the cost” retrieve the same content, which keyword search fails at.
• Vocabulary mismatch. A visitor saying “chatbot” finds content saying “assistant”.
• Conceptual grouping. Related content clusters without anyone tagging it.
• Multilingual, with the right model. A question in one language can retrieve content in another.

Choosing dimensions and models

Higher dimensional embeddings capture more nuance and cost more to store and search. For a website assistant with a few thousand chunks, this is not the constraint people expect it to be. The index is small and search is fast either way.

The more consequential choice is consistency: the same model must embed your content and your queries. Mixing models produces vectors in different spaces and the similarity scores become meaningless. If you change embedding models, you must reindex everything. This is the migration cost people forget when they plan an upgrade.

A useful mental model

Treat embedding search as a fast, fuzzy first pass that narrows thousands of chunks to a handful. It is a filter, not a decision. If precision matters for a particular question type, add a second stage that applies more expensive logic to those few candidates, whether that is keyword matching, reranking or a rule.

What an embedding actually encodes

An embedding is a list of numbers positioned so that texts with similar meaning end up near each other. That is the whole of it, and the limits follow directly from that definition.

It encodes meaning as expressed in the text. It does not encode truth, recency, authority, or whether the page was superseded last March. Two passages that say opposite things about the same subject can sit close together, because they are about the same subject.

This is worth internalising because most retrieval surprises are people expecting an embedding to carry information it was never designed to hold.

Where similarity misleads

Negation is the classic case. A passage saying the integration is supported and one saying it is not supported are close in embedding space, because almost every word matches. A question about whether it is supported will retrieve both, and the ranking between them is close to arbitrary.

Specificity is the second. A short precise answer and a long vague discussion of the same topic will often rank with the vague one higher, because it contains more of the question's vocabulary. More words that resemble the question is not more answer.

Named entities are the third. Product names, SKUs and error codes are exactly where keyword matching outperforms semantic search, which is why serious systems run both and merge the results rather than choosing.

What to do about it

Run hybrid retrieval. Keyword search catches exact identifiers, semantic search catches paraphrase, and merging the two covers the failure modes each has alone. This is the single highest value change available once chunking is sensible.

Add a reranking pass if answers are nearly right. Reranking reads the question and each candidate passage together with a slower model, which fixes the common case where the correct passage was retrieved but ranked fourth and therefore never reached the prompt.

Handle recency outside the embedding, with metadata. Store a date with each passage and prefer newer ones when two are close. Nothing in the vector will do this for you.

Choosing an embedding model, briefly

The choice matters less than the surrounding decisions, which is worth saying because it is an easy thing to spend a week on.

Dimensionality trades index size against precision, and the difference between a good general model and a slightly better one is smaller than the difference between structural chunking and character chunking. Fix the chunking first.

The one property worth checking is whether the model handles your domain vocabulary. A general model trained on web text handles most business language well and handles specialist identifiers, part numbers and internal jargon poorly. That is the gap hybrid retrieval covers, which is another argument for running keyword search alongside rather than choosing between them.

Changing embedding model later means reindexing everything, since vectors from different models are not comparable. That is a migration rather than a setting change, and it is the main reason to think about it once rather than iterate on it.

What to remember

An embedding encodes what a passage is about, and nothing about whether it is true, current or authoritative. Every retrieval surprise traces back to expecting one of those three.

Terms in this articleRetrieval Augmented GenerationEmbeddingVector DatabaseIndex