Two retrievers are better than one.

What is a hybrid search in RAG?

Hybrid search is retrieval that combines a sparse keyword method, almost always BM25, with dense vector search over embeddings, and merges the two ranked lists into one before the language model sees anything. The point is coverage: a query like 'error 0x80070005 after update' needs exact-token matching, while 'why does my laptop refuse to install updates' needs meaning. One retriever handles each; the fusion handles both.

What is BM25 in RAG?

BM25 is the classic keyword ranking function used by search engines such as Elasticsearch and Lucene. It scores a document for a query by how often the query terms appear in it (with diminishing returns), how rare those terms are across the whole collection, and how long the document is. In RAG it is the sparse retriever: fast, needs no model, and precise on terms an embedding model may blur, such as part numbers, acronyms and proper names.

How to implement hybrid search in RAG

• Index every chunk twice: as an embedding in a vector store and as tokens in a BM25 index (rank_bm25 in Python, or Postgres full-text search, or Elasticsearch).
• For each query, fetch the top twenty from both.
• Fuse with reciprocal-rank fusion: each document scores the sum over lists of 1 / (60 + rank). Sort by that score.
• Re-rank the fused top twenty with a cross-encoder and keep the top five.
• Measure recall at k against a labelled question set before and after; keep what improves it.

Here is reciprocal-rank fusion with numbers. Suppose passage A is ranked first by BM25 and fourth by vector search, and passage B is ranked second by vector search and absent from the BM25 list. With the usual constant of 60, A scores 1/61 plus 1/64, about 0.0320; B scores 1/62, about 0.0161. A wins because both retrievers liked it, even though neither put it at the top of everything. That property, rewarding agreement, is why the method is robust without tuning and why it beats naive score averaging, where BM25 scores in the tens and cosine scores below one cannot be added without normalisation. The whole function is a loop over two lists and a dictionary; write it yourself once so you understand what your vector database is doing when it offers 'hybrid' as a checkbox.

Difference between BM25 and vector search

BM25 matches surface form: the exact tokens, their frequency and rarity. Vector search matches meaning: embeddings that sit close together. BM25 cannot see that 'refund' and 'money back' are related; vector search can. Vector search can rank a passage about the right topic above the one with the exact figure the user typed; BM25 would not. BM25 is cheaper and explainable; vector search needs an embedding model and an index.

Does BM25 use embeddings?

No. BM25 works on tokens, typically lower-cased, stemmed words, and keeps an inverted index from each token to the documents containing it. There is no learned model and no vector. That is why it is the ideal complement to embedding-based retrieval: its errors are different, so fusing the two cancels more mistakes than either makes alone. Some systems also learn sparse vectors (SPLADE), but plain BM25 remains the standard baseline.

What is semantic search vs keyword search?

Keyword search finds documents containing the words you typed; semantic search finds documents meaning what you typed. Keyword search is precise and transparent; semantic search is forgiving and intent-aware. A help centre needs semantic search because people describe problems in their own words; a parts catalogue needs keyword search because the code is the query. Most real collections need both, which is hybrid search.

Difference between semantic search and hybrid search

Semantic search is one retriever: vectors only. Hybrid search is two retrievers whose results are fused: vectors plus BM25. Hybrid is a superset that costs one more index and one more query per request. In practice the extra cost is small and the gain is large on any collection that contains names, codes, dates or jargon, which is nearly every business collection. If you can only afford one, start semantic; if you can afford both, hybrid.

How hybrid search works in RAG

The question arrives; it is embedded for the vector index and tokenised for BM25; both indexes return their top candidates with ranks; reciprocal-rank fusion assigns each candidate a score based on its rank in each list, which rewards documents that both retrievers liked; the merged list is re-ranked by a cross-encoder that reads question and passage together; the top handful go into the prompt. In RAG.NextUpgrad a confidence gate then scores that evidence before any answer is generated.

What is a hybrid search strategy in RAG retrieval?

A strategy decides how much each retriever counts and how many candidates each contributes. Reciprocal-rank fusion is rank-based and needs no tuning, which is why I default to it. Weighted score fusion lets you favour keywords for a catalogue or vectors for prose, but requires normalising scores. Some pipelines route by query type: if the query looks like a code or an ID, lean on BM25. Whatever the strategy, keep a labelled test set and let recall decide.

What is hybrid search in vector database?

Several vector databases now offer hybrid search natively: Qdrant, Weaviate and Elasticsearch can store a dense vector and a sparse representation per document and fuse them inside one query, and Postgres can combine pgvector with its own full-text search in SQL. That removes the need to run two systems. The principle is unchanged: two signals, fused, then re-ranked, and measured against real questions.