RAG (Retrieval-Augmented Generation) is an architecture that grounds a large language model's answers in documents retrieved at question time: instead of answering purely from what it memorized during training, the system searches a knowledge source, places the most relevant passages into the model's context, and generates an answer built on - and ideally citing - that retrieved material. The name unpacks itself: retrieval finds the evidence, augmented means the evidence is added to the prompt, and generation is the model writing the answer on top of it. This page is the working definition, the pipeline in plain steps, and the reason the pattern matters well beyond engineering teams.
The definition
The term comes from a 2020 research paper by Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, which proposed pairing a generative model with a retriever over an external document index. The idea outgrew the paper: today "RAG" names the whole family of systems where an LLM answers with retrieved evidence in its context window rather than from parametric memory alone. The useful mental model is a division of labor. The model contributes reasoning, language and synthesis - skills that generalize. The retrieval layer contributes knowledge - facts that change, belong to you, or appeared after training ended. Because the knowledge lives in a searchable store instead of the model's weights, you can update it by updating documents, restrict it by permission, and audit an answer by reading exactly what was retrieved to produce it. That last property, verifiability, is what makes Retrieval-Augmented Generation the default architecture for any system whose answers must be checked.
Why LLMs need retrieval
Three limits of a standalone model make retrieval necessary rather than optional. The knowledge cutoff: a model knows nothing after its training data ends, so any question about this quarter's pricing, last week's release or today's inventory is unanswerable from weights alone. Private data: your contracts, docs, tickets and CRM were never in the training set and never should be - retrieval is how a general model answers over knowledge it was deliberately never trained on. Hallucination: when a model lacks the fact, it often produces a fluent, confident, wrong sentence, because generating plausible text is precisely what it was trained to do. Retrieval attacks all three at once: the store is as fresh as its last update, it can hold anything you are permitted to index, and grounding the answer in quoted passages gives both the model and the reader something to check. It is honest to say RAG reduces hallucination rather than eliminates it - a model can still misread or overrun its evidence - which is why serious pipelines cite sources and let humans verify the load-bearing claims. Whether a given problem calls for retrieval or for training knowledge into the weights is its own decision - our RAG vs fine-tuning comparison walks that trade-off honestly.
How a RAG pipeline works, step by step
| Stage | What happens | What goes wrong here |
|---|---|---|
| Ingest | Documents are collected and parsed from their sources | Stale sources; unparseable formats; missing permissions |
| Chunk | Documents are split into passages sized for retrieval | Splits that sever a fact from its context |
| Embed | Each chunk becomes a vector capturing its meaning | An embedding model mismatched to the domain |
| Retrieve | The question is embedded and the nearest chunks are fetched | Semantically similar but factually irrelevant hits |
| Rerank | A second pass reorders candidates by true relevance | Skipping it and trusting raw similarity scores |
| Generate | The LLM writes the answer from the passages, with citations | Answers that drift beyond what the evidence supports |
Two things about the table are worth internalizing. First, the stages before the model ever runs - ingest, chunk, embed - decide most of the quality ceiling; a brilliant model reading the wrong passages writes a brilliant wrong answer. Second, citation is a pipeline feature rather than a courtesy: because the generator saw specific passages, it can point at them, and a reader can follow the pointer. Production systems commonly retrieve with both vector similarity and classic keyword search, then rerank the merged candidates, because each method misses things the other catches. When a RAG system disappoints, the fix is almost never "a smarter model" and almost always one specific stage: fresher ingestion, saner chunking, or a reranker earning its place.
Where RAG shows up in the products your buyers use
The most consequential RAG deployment is one most people never call by name: AI assistants answering with cited sources. When ChatGPT, Perplexity, Gemini or Copilot searches the web, reads pages, and composes an answer with links attached, that is Retrieval-Augmented Generation running over the live web at enormous scale - the "knowledge store" is the indexed internet, and the retrieved chunks are pages like yours. This is why AI citations exist at all, and why they became a competitive surface: for every buyer question an assistant answers, some site's content was retrieved, and some site's was skipped. The entire discipline of answer engine optimization is, mechanically, the practice of winning the retrieve and rerank stages of somebody else's RAG pipeline - publishing content structured so the retriever finds it, the reranker prefers it, and the generator can lift it cleanly. The same pattern runs inside products: support bots grounded in help docs, sales assistants grounded in CRM notes, internal search grounded in the company wiki. Anywhere an AI product answers with sources attached, RAG is the machinery underneath.
Agentic RAG: when the agent decides what to fetch
Classic RAG is a fixed reflex: one question, one retrieval, one answer. Agentic RAG replaces the reflex with judgment. An agent reads the question, decides what it needs to know, issues one or several retrievals - possibly against different stores, possibly rewriting the query - then inspects what came back and asks the harder question: is this evidence actually sufficient? If it is thin or contradictory, the agent searches again, differently, before it writes a word. Anthropic's guidance on building effective agents frames this well: retrieval becomes one tool in a loop the model directs, alongside APIs, code and other actions. That loop is what lets agents do multi-step knowledge work - research a market, audit a funnel, reconcile a report - where the right sources are unknowable in advance, and it is the architecture underneath how AI agents execute real operational work.
Frequently asked questions
What does RAG stand for?
Retrieval-Augmented Generation. Retrieval finds relevant documents at question time, augmented means those documents are added to the model’s context, and generation is the LLM writing an answer grounded in that evidence. The term comes from a 2020 research paper by Lewis et al.
What is a RAG pipeline?
The end-to-end system that makes RAG work: ingest documents, chunk them into passages, embed the chunks as vectors, retrieve the nearest ones for a question, rerank the candidates, and generate an answer with citations. Most quality problems trace to one specific stage rather than the model.
Does RAG stop hallucinations?
It reduces them substantially by giving the model real evidence to write from, and citations let readers verify claims. It does not eliminate them - a model can still misread or overrun its sources - so well-built systems cite passages and keep humans checking load-bearing answers.
What is the difference between RAG and fine-tuning?
RAG adds knowledge at question time by retrieving documents into context; fine-tuning changes the model’s weights through additional training. RAG wins for fresh, private or verifiable knowledge; fine-tuning wins for behavior, style and format. Many production systems use both.
Why should marketers care about RAG?
Because every AI assistant that answers with cited sources is running RAG over the web. Your content is either retrieved into those answers or absent from them, and answer engine optimization is effectively the practice of winning the retrieval stage of the assistants’ pipelines.
Sources
- Lewis et al. - Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (the original RAG paper)
- Anthropic - Building effective agents
- llms.txt - the proposed standard for LLM crawler guidance
Every playbook on this blog ships as a runnable mission.
Open a workspace and the playbook library is waiting - describe the outcome and the agents carry it end to end, on your plan's monthly credits.