What are embeddings? The math behind RAG, without the math

If you’ve read anything about RAG, semantic search, or modern recommendation systems, you’ve seen the word embedding. Often without an explanation that actually helps.

Here it is in plain English.

The library card analogy

Imagine you run a giant library and you want to organise the books so that similar books end up near each other. Not by author, not by title — by meaning.

You hire a very careful librarian and ask them: for every book, give me a card that captures, numerically, what the book is about. Books with similar meanings should get similar cards. The card itself is meaningless to look at — it’s just a long list of numbers — but it perfectly captures the book’s “topic position” in some imaginary space.

That card is an embedding.

An embedding model is a function that takes any piece of text (or image, or audio) and produces a long list of numbers — typically a few hundred to a few thousand of them. The same text always produces the same numbers. Similar text produces similar numbers. That’s it.

Why this is useful

Once everything is represented as numbers, you can do math on it. Specifically, you can ask: how similar are these two texts?

The answer is just the distance between their two embeddings. Close together = similar meaning. Far apart = unrelated.

This sounds basic. It’s a superpower:

• Search by meaning, not by keywords. A user asks “how do I cancel my subscription?” and the system finds the help article titled “managing your account” — because the meanings are close, even though the words are different.
• Find similar items. Recommend articles, products, or documents based on what someone is currently looking at.
• Cluster automatically. Group documents, tickets, or user queries by topic without anyone manually tagging them.
• Detect duplicates. Find near-identical content even when the wording is different.

Embeddings power RAG

A RAG system is essentially:

1. Chop your documents into chunks.
2. Get an embedding for every chunk. Store them.
3. When a user asks a question, get an embedding for the question.
4. Find the chunks whose embeddings are closest to the question’s embedding.
5. Hand those chunks to the language model as context.

Steps 1, 2, and 4 are where embeddings live. They are the reason RAG can match a question to relevant content without anyone writing rules.

A few things embeddings are bad at

It’s worth knowing the failure modes:

• Exact matches. Embeddings are about meaning, not strings. Searching for a product code “ACME-9000” by embedding alone may miss the document that literally contains it. (This is why hybrid search — embeddings + keyword — usually wins.)
• Negation. “Documents about cats” and “documents not about cats” can end up with similar embeddings. The model captures topic, not logic.
• Rare or domain-specific terms. General-purpose embedding models don’t know your internal jargon, product names, or codes. The embedding for an unknown term can be unhelpful.
• Long documents. Most embedding models have a context limit. A whole 50-page contract embedded as one vector loses fidelity. Chunking matters.

How to pick an embedding model

There are dozens. A few practical pointers:

• Open vs. hosted. OpenAI, Cohere, Voyage, and others sell hosted embeddings. Models like BGE, E5, and Nomic are open-weight and can run on your own infrastructure.
• Dimension matters. Higher-dimensional embeddings (3072, 1536) capture more nuance but cost more storage and slightly more compute. Lower-dimensional ones (256, 384) are leaner but coarser. Most teams over-pick on dimension.
• Domain match. A general-purpose embedding may not be the best for code, biomedical text, legal text, or other specialised content. Specialised embedding models exist; they often beat generic ones on their target domain.
• Measure, don’t guess. Build a small evaluation set of “this query should match these documents” and test 2-3 candidate embedding models on it. The answer is often counter-intuitive.

A useful mental model

When you hear someone say “we used embeddings to find similar items” or “we’re switching to a better embedding model,” what they mean in plain terms is:

• Embeddings = compressed numerical fingerprints of meaning.
• Better embedding model = a model whose fingerprints group similar things more accurately.
• Vector database = a place to store many of these fingerprints and quickly find the closest matches.

That’s almost all you need to know to make sense of the rest of the RAG conversation.

Further reading:

• What is RAG, and when should you use it?
• Different ways to build a RAG system