A confidence gate checks the top retrieved result's similarity score against a threshold before the language model is ever called. Fail the threshold, and the system returns a fixed refusal instead of letting the model improvise past its sources.
Why this has to happen before generation
Asking a model to self-assess its own confidence after answering means the system that might hallucinate is grading its own hallucination. A confidence gate checks the evidence, not the model's opinion of its own output — a check the model can't talk its way past.
Setting the threshold
Too high, and the system refuses paraphrased questions it could actually answer. Too low, and you're back to confident nonsense. There's no universal number — it has to be tuned against a real evaluation set of your own documents and your own users' phrasing.
The unexpected benefit: it's also a cost feature
Every refused call is a language-model call you didn't pay for. A well-tuned confidence gate reduces both hallucination and your API bill, which is a rare case of the safer choice also being the cheaper one.