Search by meaning, not spelling.

What is semantic search?

Semantic search is retrieval that matches on meaning. The query and every document are converted into embeddings, vectors in which similar meanings sit close together, and results are ranked by how close they are to the query. Ask 'how do I get my money back' and it finds the refund policy even though the words 'money back' never appear in it. That is the behaviour users expect from a search box in 2026.

How semantic search works

At index time, each document (or chunk of one) is passed through an embedding model and the resulting vector is stored. At query time the question is embedded with the same model, the store returns the nearest vectors by cosine similarity, usually through an approximate index such as HNSW so it stays fast at scale, and the matching documents are returned in order. Optionally a re-ranker rescores the top candidates for precision.

How to implement semantic search

• Pick an embedding model (an open BGE or E5 model, or a hosted one) and keep it fixed; changing it means re-indexing.
• Chunk documents by structure, embed each chunk, store vector plus text plus metadata in pgvector, Qdrant or FAISS.
• Embed the query, fetch the top twenty by cosine similarity, apply metadata filters in the same query.
• Add a BM25 index and fuse the two result lists with reciprocal-rank fusion.
• Re-rank the fused top twenty with a cross-encoder and show the top five with their source.

Two additions make a noticeable difference once the basics work. Query rewriting: short or vague queries embed poorly, so have a small model expand 'refund?' into 'how do I request a refund for an order' before embedding, or generate a hypothetical answer and embed that. Context prefixes: prepend the document title and section heading to each chunk before embedding so the vector carries where the text sits, not only what it says. Then measure. Take fifty real queries, label the passage each should return, and compute recall at five for vector-only, keyword-only and hybrid. The table that produces is the most persuasive artefact you can show a team that is arguing about search from intuition.

What is semantic search in RAG?

In retrieval-augmented generation, semantic search is the retriever: it chooses the passages the language model is allowed to read. If retrieval misses, no amount of prompting fixes the answer. In RAG.NextUpgrad semantic (dense) retrieval runs alongside BM25 keyword retrieval, the lists are fused and re-ranked, and a confidence gate checks the evidence before the model answers. Retrieval quality is measured with recall at k on a real question set.

Lexical vs semantic search

Lexical (keyword) search matches tokens: BM25 scores documents by how often the query's words appear, weighted by rarity. It is exact, explainable and excellent for names, part numbers and jargon. Semantic search matches meaning and handles paraphrase and synonyms, but can return something that is about the topic without containing the specific fact. Each fails where the other is strong, which is the whole argument for combining them.

Difference between vector search and semantic search

Vector search is the mechanism: nearest-neighbour search over vectors. Semantic search is the goal: finding things by meaning. Semantic search is almost always implemented with vector search over embeddings, so people use the terms interchangeably, but vector search can also run over non-semantic vectors (image colour histograms, say) and semantic search can include a keyword component. Mechanism versus intent.

Do vector databases use cosine similarity?

Most do by default, or an equivalent. Cosine similarity measures the angle between two vectors, ignoring their length, so it compares direction, which is where meaning lives after normalisation. Dot product and Euclidean distance are also offered; on unit-normalised vectors cosine and dot product rank identically and dot product is cheaper. Pick the metric your embedding model was trained for and keep it consistent between indexing and querying.

Is semantic search deterministic?

The embedding step is deterministic for a fixed model: the same text always produces the same vector. The search is deterministic for an exact scan and nearly so for approximate indexes like HNSW, which can return slightly different neighbour sets after inserts or with different index parameters. What is not deterministic is a language model summarising the results afterwards, so if you need reproducible answers, pin the model, the index and the prompt, and test.

What is the difference between retrieval augmented generation and semantic search?

Semantic search returns documents; RAG returns an answer written from them. RAG contains a semantic search (often a hybrid one) as its retrieval stage, then adds a language model that reads the retrieved passages and composes a reply with citations. If users want to read sources themselves, a good search is enough. If they want a direct answer with the sources attached, you need RAG, and it is only as good as the search inside it.

Why do embeddings improve semantic search?

Because the embedding model has already learned, from enormous amounts of text, which phrasings mean the same thing. That knowledge is baked into where each text lands in vector space, so similarity becomes a geometry problem instead of a vocabulary problem. Better embedding models, and embedding models fine-tuned on your own query-document pairs, move the right documents closer to the right questions, which shows up directly as higher recall.