What is RAG (retrieval-augmented generation)? And when it goes wrong

What is RAG?

RAG stands for retrieval-augmented generation. It describes an assistant that looks up the relevant documents first, then answers from them. A question arrives, the assistant searches the files you connected it to (a policy folder, a shared drive, a help center), pulls the few passages that match, and writes the reply from those passages. Good setups show which file each answer came from.

The confusion worth clearing up: this is not training an assistant on your data. Nothing gets memorized. The files get searched fresh on every question, so a price list you add on Tuesday shows up in Wednesday's answers with no retraining and no waiting on a vendor.

Where it goes wrong

The lookup step causes most of the bad answers, and the writing takes the blame. Leave last year's benefits policy in the same folder as this year's and the search can hand back the old one. The reply reads just as confidently and still names a real file. Engineers hit the same thing when a question about version 15 of a product gets answered out of the version 13 documents. A citation proves which file was read, not that the answer matches what the file says. Archive the stale copies, and open the cited source before an answer goes to a customer.

What you actually need

Off-the-shelf tools cover a lot of this. Google's NotebookLM holds 50 sources per notebook on the free plan, 100 on Plus and 300 on Pro. If your answers come from a few dozen documents that somebody keeps tidy, that is enough. Set up something sturdier when answers have to pull from several systems at once, or when the documents change weekly and nobody is pruning the old versions.