Retrieval is only as good as the query it receives, and users write queries like "that thing about refunds after 30 days for the premium plan". The document says "Refund eligibility: Premium subscribers, 30-day window". Bridging that gap before retrieval is query rewriting, and it is cheaper than most people expect.

Expansion: more words for the same meaning

Ask a small model to produce two or three alternative phrasings of the question, run retrieval on all of them, and merge with Reciprocal Rank Fusion — the same merge used in hybrid retrieval. Expansion helps most when the corpus uses formal vocabulary and users do not.

Decomposition: one question at a time

"Compare the refund policy for premium and basic plans" is two retrievals. Split it into sub-questions, retrieve for each, and give the model both contexts labelled. Without decomposition, the retriever returns whichever plan is mentioned more often and the comparison is invented.

HyDE: retrieve with a guess

Hypothetical Document Embeddings has the model write a short, plausible answer first, then embeds that answer to search with. The hypothetical text is closer in style to real passages than the question was, so it often retrieves better. The catch: if the model's guess is wrong in vocabulary, retrieval follows it off a cliff. Use it alongside the original query, never instead of it.

How to know if it helped

  1. Run your evaluation set with and without each rewrite step and compare recall@5.
  2. Log the rewritten queries in production and read a sample weekly — you will find rewrites that change the meaning.
  3. Measure added latency; an extra model call before retrieval is fine for chat, not for autocomplete.

What I actually run

For most document-QA products: hybrid retrieval on the original query plus one expansion, decomposition only when a classifier flags a multi-part question, and no HyDE by default. Simple systems with good chunking beat clever ones with bad chunks, every time.

— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.