Retrieval-augmented generation, plain-English definition from TheAICommand
← Glossary
Definition

What is retrieval-augmented generation (RAG)?

Retrieval-augmented generation pairs a language model with a separate search over a defined knowledge base, so answers are grounded in named documents rather than model memory. The term comes from a 2020 paper by Lewis and colleagues. For Australian regulated work it moves the compliance question from what the model knows to what the index holds and who can reach it.

Quick answer

Retrieval-augmented generation, or RAG, is a technique that pairs a language model with a separate search over a defined knowledge base. The system retrieves relevant passages for a question and supplies them to the model as context, so answers are grounded in named documents rather than model memory alone.

What is retrieval-augmented generation?

Retrieval-augmented generation is an architecture, not a model. A language model on its own answers from parameters fixed at training time. A RAG system inserts a retrieval step in front of the model: the question is used to search a defined corpus, the most relevant passages come back, and those passages sit in the model's context before it writes anything.

NIST's adversarial machine learning taxonomy defines RAG as a generative AI system in which a model is paired with a separate information retrieval system, or knowledge base, and makes the practical consequence plain. RAG lets the internal knowledge of a model be changed without retraining. Update the document, and the next answer changes.

The term comes from a paper submitted in May 2020 by Patrick Lewis and colleagues, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. It framed the problem as parametric memory against non-parametric memory: what a model has absorbed into its weights, versus an external index it can look things up in. Their implementation paired a pre-trained sequence-to-sequence model with a dense vector index of Wikipedia reached through a neural retriever. They reported state of the art results on three open domain question answering tasks, and language that was more specific and more factual than a parametric-only baseline.

The 2023 survey by Gao and colleagues groups the work that followed into three paradigms, Naive RAG, Advanced RAG and Modular RAG. The direction of travel is more engineering around retrieval, not less.

Who uses RAG, and where does it turn up at work?

Almost every enterprise assistant that answers from internal documents is a RAG system, whether or not the vendor uses the term. AWS sets out the pipeline in four stages: create the external data as vector representations in a database, retrieve relevant information through a relevancy search, augment the prompt by adding the retrieved data in context, and update the external data asynchronously through real-time processes or periodic batch, so it does not go stale.

In practice that covers policy and procedure assistants, contract and claim file lookups, internal help desks, and the chat-with-your-documents features now bundled into productivity suites. The reason the pattern spread so fast is cost. Retraining a foundation model to teach it your procedures is expensive and slow. Pointing a retriever at a document store is neither.

Where does RAG fit in Australian regulated work?

RAG moves the governance question from what the model knows to what the index contains and who can reach it. Three consequences follow.

Retrieval respects whatever access control you enforce at retrieval time, and nothing else. If the index is flat while permissions live only in the source system, a well-formed question can surface a document the person asking was never entitled to read.

Provenance becomes available. Because the system can cite the passages it used, a reviewer can check the answer against the source. That is the difference between a tool a practitioner can defend in a review and one they cannot.

The index becomes an attack surface. NIST identifies RAG knowledge bases and internet-connected agents as the classic settings for indirect prompt injection, where a poisoned document carries instructions the model may act on.

For APRA-regulated entities, an externally hosted RAG service supporting a critical operation can fall inside the service provider obligations of CPS 230, and the index itself is an information asset under CPS 234. None of that is triggered by the word RAG. It is triggered by what the system does and what it holds.

What should practitioners do about RAG?

Treat the index as a record, not a convenience. Document what is in it, where each document came from, how often it refreshes, and what happens when a source is superseded. A stale index produces confidently wrong answers with citations attached, which is worse than no answer at all.

Enforce permissions at retrieval rather than at display. Test the failure mode where retrieval returns nothing relevant, and confirm the system says so instead of filling the gap from model memory. Keep a human in the loop wherever an output becomes a decision, a determination or advice, and require that reviewer to open at least one cited source rather than trusting that a citation exists.

Do not oversell it. RAG reduces fabrication; it does not eliminate it. The model can still misread a retrieved passage or blend two sources into a claim neither of them makes. Where an agentic system is doing the retrieving and then acting on the result, the stakes rise again.

Bottom line

Retrieval-augmented generation is an architecture rather than a model, inserting a search over a defined corpus in front of the language model so answers are grounded in named documents instead of model memory alone. That shifts the governance question from what the model knows to what the index contains and who can reach it, which is the part most often missed in Australian regulated work. Retrieval respects only the access control enforced at retrieval time, so a flat index sitting behind source-system permissions lets a well-formed question surface a document the person asking was never entitled to read, and the index is also a durable place to hide an instruction for an indirect injection. Treat the index as a record with documented provenance and refresh, test what the system does when retrieval returns nothing useful, and require the human reviewing an output to open at least one cited source rather than trusting that a citation exists.

TheAICommand. Intelligence, At Your Command.*

TheAICommand. Intelligence, At Your Command.

Frequently asked questions

Does RAG stop AI from hallucinating?
No. RAG reduces fabrication by grounding answers in retrieved passages, and the 2020 paper reported more factual language than a parametric-only baseline. The model can still misread a passage, blend sources, or answer confidently when retrieval returns nothing useful. Citations make errors easier to catch, not impossible.
What is the difference between RAG and fine-tuning?
Fine-tuning changes the model weights by training on additional examples. RAG leaves the model unchanged and supplies information at query time from an external index. NIST notes RAG lets the internal knowledge of a model be modified without retraining, so updating a document updates the answer immediately.
Is RAG safe for confidential or regulated data?
It depends on the controls around the index, not on RAG itself. Retrieval can surface any document the permissions allow, so access control must be enforced at retrieval time. NIST also identifies RAG knowledge bases as a route for indirect prompt injection, where poisoned documents carry hidden instructions.
Where did the term RAG come from?
From a paper submitted in May 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, by Patrick Lewis and colleagues. It described models that combine pre-trained parametric memory, a sequence-to-sequence model, with non-parametric memory, a dense vector index of Wikipedia accessed by a neural retriever.
What does a RAG pipeline do at query time?
AWS describes four stages. Create external data as vector representations in a database, retrieve relevant information by relevancy search, augment the prompt by adding the retrieved data in context, and update the external data asynchronously through real-time processes or periodic batch, so it does not go stale.

Primary sources

← Back to the glossary

General information and education only. Not legal, compliance, financial, or professional advice. Always confirm obligations against the primary source and current regulator guidance.