Classify the failure
- Unsupported fact: the source does not contain the claim.
- Wrong interpretation: the source exists but the answer misreads it.
- Invented citation: the link or document does not support the sentence.
- Fabricated number: a statistic, price, date, or calculation is wrong.
- Overconfident uncertainty: the system should have refused or asked for clarification.
Build a detection loop
Create a reviewed test set, run the system against it, compare each material claim with evidence, and record the failure type. In production, sample outputs by risk and feed confirmed failures back into the evaluation set.
Signals worth monitoring
| Signal | What it may reveal |
|---|---|
| High correction rate | The system produces fluent but unusable output |
| Citation clicks that do not support claims | Grounding or source selection failure |
| Frequent retries | The task or prompt is unstable |
| Answers with no source when one is required | Missing evidence or weak refusal behavior |
| Different answers to the same input | Uncontrolled randomness or context drift |