Also known as: RAG · retrieval augmented generation
Retrieval-augmented generation (RAG) is a technique in which relevant documents are retrieved from an external knowledge source and supplied to a language model as context, so that its response is grounded in specific retrieved material rather than parametric memory alone.
RAG addresses several limitations simultaneously: it supplies knowledge more recent than the training cutoff, allows citation of sources, keeps proprietary data out of training, and reduces hallucination by giving the model actual text to work from.
Production experience has shown retrieval quality, rather than model choice, to be the dominant factor in whether a RAG system works. The characteristic failure is not poor reasoning over correct documents but confident reasoning over irrelevant ones that scored highly on similarity.
Hallucination
A hallucination is output from an AI model that is fluent and confident but factually incorrect or entirely fabricated, such as an invented citation, a non-existent API method, or a plausible-sounding but false statistic.
Embedding
An embedding is a numerical vector representation of text, images, or other data, positioned in a high-dimensional space such that semantically similar items sit close together.
Vector Database
A vector database is a data store optimised for indexing and querying high-dimensional embedding vectors, typically using approximate nearest neighbour algorithms to find the most semantically similar items quickly.
Grounding
Grounding is the practice of tying a model's output to specific verifiable source material, so that claims can be traced to documents rather than resting on the model's parametric memory.
Context Window
The context window is the maximum number of tokens a language model can consider at once, encompassing both the input prompt and the generated output.
AI Agent
An AI agent is a system that uses a language model to plan and execute multi-step tasks by calling external tools — searching, running code, querying APIs — and iterating on the results, rather than producing a single response.