What is RAG (retrieval-augmented generation)?

RAG, or retrieval-augmented generation, is a technique that lets an LLM answer using specific documents it fetches at the moment you ask, instead of relying only on what it memorized during training. In plain terms: before the model responds, the system searches a knowledge source — your company docs, a database, a set of PDFs — pulls the most relevant passages, and hands them to the model as context. The LLM then writes its answer grounded in that retrieved material.

How does RAG actually work?

A RAG system runs a quick three-step process every time you ask something:

1. Retrieve — it takes your question, searches the knowledge source, and pulls back the handful of passages most relevant to it.
2. Augment — it stuffs those passages into the prompt alongside your question, essentially saying "here's the reference material, now answer."
3. Generate — the LLM writes its response from that supplied text rather than from memory alone.

The search step usually relies on embeddings, which match passages by meaning rather than exact keywords — so a question about "time off" can still find a document titled "vacation policy."

What problems does RAG solve?

It fixes two big weaknesses of a plain LLM. The first is knowledge gaps: a model's training has a cutoff date and knows nothing about your private data, but RAG can feed it fresh or proprietary information on demand. The second is hallucination — because the model is answering from real supplied text, it's far less likely to invent facts, and it can even cite the source a claim came from.

You get answers tied to material you control, not just the model's fuzzy memory.

Why is RAG so widely used?

It has become one of the most common ways companies build AI on their own data. Customer-support bots, internal search, and document assistants are frequently RAG systems under the hood. The appeal is practical: it's cheaper and faster than retraining a model every time your data changes, it keeps answers grounded in sources you own, and when a document updates, the system simply retrieves the new version — no retraining required.