Also known as: context length · context size
The context window is the maximum number of tokens a language model can consider at once, encompassing both the input prompt and the generated output.
Context windows have expanded dramatically, from a few thousand tokens to hundreds of thousands or more. This has been genuinely transformative for tasks involving long documents or large codebases that previously required chunking.
Larger windows do not guarantee uniform attention across them. Models frequently exhibit degraded recall of material in the middle of a very long context — the 'lost in the middle' effect — which is why careful retrieval of relevant passages often outperforms simply pasting everything in.
Token
A token is the atomic unit of text that a language model processes — typically a common word, a word fragment, or a punctuation mark — produced by splitting input text through a tokenizer before the model sees it.
Large Language Model
A large language model (LLM) is a neural network trained on vast quantities of text to predict likely continuations of a sequence, producing systems capable of generating coherent language, answering questions, and writing code.
Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) is a technique in which relevant documents are retrieved from an external knowledge source and supplied to a language model as context, so that its response is grounded in specific retrieved material rather than parametric memory alone.
Transformer
The transformer is a neural network architecture, introduced in 2017, that uses self-attention to weigh the relevance of every input token against every other, and which underpins essentially all modern large language models.
Prompt Engineering
Prompt engineering is the practice of structuring input to a language model — through instructions, examples, formatting, and reasoning scaffolds — to reliably obtain the desired output.