How / What / When

How / Answer

How to write better prompts for a RAG system

A good RAG prompt makes the evidence boundary explicit: what the model may use, how it should cite it, what it should do when evidence is missing, and what the final answer must look like.

For builders improving grounded answers without hiding retrieval failures behind confident prose. · Reviewed 2026-09-29

Put the job before the context

Tell the model what the answer must help the user decide or do. Then define the source boundary. If you paste documents first and instructions second, the model may treat text inside a document as an instruction, which is also a prompt-injection risk.

A useful prompt shape

  • Role: what kind of assistant is this?
  • Task: what question must be answered?
  • Evidence: which retrieved passages may be used?
  • Boundary: what must happen when the passages do not support an answer?
  • Citations: how should source IDs or links appear?
  • Output: what fields, length, tone, and next action are required?

The missing-evidence instruction

Write an explicit fallback such as: “If the supplied sources do not answer the question, say that the evidence is insufficient and name the missing information. Do not fill the gap from general knowledge.” Test this instruction with questions that should fail.

Keep the prompt testable

Use a small set of expected answers and failure cases. Change one part of the prompt at a time. If you cannot explain why a change improved the result, you are tuning by anecdote rather than improving the system.

Keep learning

Get one useful AI answer each week.