Fluent models are rewarded for producing an answer. Good products sometimes need them to stop.
“I don’t know” is not one behavior. The system may lack context, face conflicting evidence, receive an ambiguous request, or encounter a task outside its authority. Each case deserves a different recovery.
Define Abstention Conditions
Do not rely on the model to feel uncertain. Give it observable reasons to defer:
- retrieval returned no relevant evidence;
- required fields are missing;
- sources conflict;
- a tool failed;
- the requested action exceeds permission;
- or an evaluator flags unsupported claims.
Deterministic checks should trigger abstention where possible. For semantic uncertainty, use clear instructions and evaluate examples that should answer, clarify, or refuse.
Make Stopping Useful
A dead-end refusal frustrates users. A useful abstention explains the missing piece and offers the next action:
I cannot confirm the refund window from the documents available. Choose your plan name, or contact support for a verified answer.
That response protects trust and preserves momentum.
Measure Both Sides
If you optimize only for fewer hallucinations, the system may refuse everything. Track false answers and unnecessary refusals. Your evaluation set needs answerable, unanswerable, ambiguous, and conflicting cases.
The target is calibrated behavior: answer when evidence is sufficient, clarify when input is incomplete, and defer when the risk of guessing is too high.
An AI product earns trust not by sounding certain. It earns trust by behaving appropriately when certainty is unavailable.