A concrete example
Imagine a SaaS company with a help center, pricing page, and internal escalation guide. A customer asks, “Can I pause my annual plan for two months?” The model should not guess. A RAG system retrieves the current billing rule, passes that passage to the model, and returns an answer with a link to the policy. If the policy is missing, the safest answer is “I can’t verify that from the available documents,” not a confident invention.
How the pieces fit together
- The question is converted into a search query.
- A retriever finds relevant chunks from approved documents.
- The application decides which chunks are trustworthy and recent enough.
- The model writes an answer from that context.
- The product shows citations or supporting passages when the user needs to verify the answer.
What RAG does not fix
RAG is not a truth machine. It cannot repair a policy that is contradictory, retrieve a document that was never indexed, or stop a model from misreading an ambiguous passage. In practice, the document pipeline and the evaluation set matter as much as the model.
When RAG is the sensible choice
Use it when your knowledge changes, belongs to your organization, or must be traceable. It is a good fit for support documentation, product manuals, internal policy, and research collections. It is a weaker fit when the real problem is that the model cannot follow a stable output format or tone.
Minimum quality check before launch
- Test a question whose answer changed last month.
- Test a question that has no answer in the source set.
- Test conflicting documents and stale versions.
- Check whether citations support the exact sentence, not merely the general topic.
- Measure retrieval errors separately from writing errors.