Key takeaways
- RAG retrieves relevant material at answer time and provides it to a generative model.
- Retrieval quality and answer faithfulness are separate things to evaluate.
- A citation is useful only when the cited material supports the particular claim.
Overview
A typical system indexes documents, retrieves candidate passages and passes selected context to the model. Retrieval and generation can fail independently: the right document may not be found, or the model may misstate a retrieved fact. Source permissions, document freshness and traceable citations are therefore part of the design, not optional presentation details.
How it works
Prepare and index source material while retaining document identifiers and access rules.
Retrieve and rank passages relevant to the question.
Generate an answer grounded in the selected context and expose supporting sources.
Follow the evidence through the pipeline
Imagine a sales assistant answering which industries a company serves. A retrieval stage searches approved company materials; the generation stage produces an answer using the selected passages. If retrieval returns an unrelated subsidiary page, the model may faithfully summarize the wrong company. If retrieval returns the correct page but the answer adds unsupported industries, the failure is in generation or validation.
Keep enough context around retrieved passages to interpret them: document title, entity, date and section can matter as much as the sentence itself. An isolated line saying “available next quarter” is not useful without knowing when it was written. Access controls must apply before retrieval so private material cannot enter an answer for an unauthorized user.
| Stage | Useful check | Example failure |
|---|---|---|
| Retrieval | Did relevant authorized evidence appear? | The correct account document was absent |
| Selection | Were the strongest passages used? | An outdated page displaced a current one |
| Generation | Does each factual claim follow the evidence? | The answer invented an unsupported capability |
Freshness is an indexing decision
Adding retrieval does not make information automatically current. A system searching last year’s index will still retrieve last year’s evidence. Define how documents enter the index, how updates replace old versions and how deletions propagate. For fast-changing information, record the observation date and decide whether to refresh the source during the task.
The same applies to changed permissions. A document that was accessible during indexing may later become restricted. Design the retrieval boundary around current authorization rather than assuming an old index entry grants access forever. When a source is removed, prevent stale derived chunks from continuing to appear in results.
Evaluate answers that should remain unknown
Include questions that the approved corpus cannot answer. A good result may say that the evidence is missing and identify what would resolve the question. If every evaluation question has an answer in the corpus, the test will not reveal whether the system invents facts when retrieval fails.
Score relevance of retrieved passages, support for individual claims and the usefulness of the final response separately. An illustrative ten-question review may find nine relevant retrievals but only seven fully supported answers; reporting only retrieval success hides two generation failures. Add conflicting documents and ambiguous entity names to the review set so citations are checked for substance rather than appearance.
What this looks like in practice
A sales assistant retrieves an approved product specification and a current pricing policy before drafting an answer. It links each material claim to the relevant document and flags a question the documents do not answer.
Examples explain the concept; they are not reported customer results.What to check
Test retrieval recall, answer correctness, citation support and permission filtering separately. Include questions whose answers are absent from the source collection.
Common mistake
Assuming that adding a vector database eliminates hallucinations. Retrieval can return irrelevant, stale or malicious content, and generation still needs evaluation.
Retrieval-augmented generation vs. AI agent
RAG describes how information is supplied to generation. An agent describes how actions are selected. An agent can use retrieval as one tool, but a RAG application need not be an agent.
Read the AI agent definition →Questions answered
What is Retrieval-augmented generation?
Retrieval-augmented generation, or RAG, supplies a language model with information retrieved from an external collection so it can generate an answer using context relevant to the request.
Is RAG the same as model training?
No. RAG generally supplies information at request time rather than changing model weights. Training and retrieval can be combined, but they solve different problems.
Do citations prove an answer is correct?
No. Check that the cited passage actually supports the claim, that it is current and that the user is allowed to access it.
Does RAG train the model on my documents?
Retrieving documents into a request is different from updating model weights through training. The surrounding product still has its own data handling and retention rules, which should be checked separately. The presence of RAG alone does not establish how a provider stores or uses submitted material.
Can RAG eliminate hallucinations?
No. It can supply relevant evidence, but retrieval can miss important material and the model can still misinterpret or extend what it sees. Require support for material claims, preserve uncertainty and evaluate unsupported questions. Citations should point to evidence that a reader can actually inspect.
References and further reading
Primary documentation and source material for this topic. Sources checked September 14, 2026; provider requirements can change.
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks ↗Lewis et al., 2020
- Prompt injection ↗OWASP GenAI Security Project
Continue reading on the blog
Explore all articles and guides →Put the concept to work.
Explore the relevant AstroFabric workflow and see how the pieces connect.
Help keep this guide useful. Suggest a correction or browse the full glossary.