Also known as: LLM · language model · foundation model
A large language model (LLM) is a neural network trained on vast quantities of text to predict likely continuations of a sequence, producing systems capable of generating coherent language, answering questions, and writing code.
Despite the sophistication of their outputs, LLMs are trained on a comparatively simple objective: predicting what token comes next. The capabilities that emerge from that objective at scale — reasoning, translation, summarisation — were not explicitly programmed and remain incompletely understood.
The practical implications of the prediction objective are important. An LLM produces text that is statistically plausible given its training, which correlates strongly but imperfectly with text that is true. This is the structural reason hallucination is a persistent property rather than a bug awaiting a fix.
Transformer
The transformer is a neural network architecture, introduced in 2017, that uses self-attention to weigh the relevance of every input token against every other, and which underpins essentially all modern large language models.
Token
A token is the atomic unit of text that a language model processes — typically a common word, a word fragment, or a punctuation mark — produced by splitting input text through a tokenizer before the model sees it.
Hallucination
A hallucination is output from an AI model that is fluent and confident but factually incorrect or entirely fabricated, such as an invented citation, a non-existent API method, or a plausible-sounding but false statistic.
Fine-Tuning
Fine-tuning is the process of further training a pre-trained model on a smaller, task-specific dataset to adapt its behaviour, style, or domain knowledge without training from scratch.
Open-Weight Model
An open-weight model is one whose trained parameters are publicly downloadable, allowing anyone to run, inspect, fine-tune, and deploy it on their own infrastructure without calling an external API.
Context Window
The context window is the maximum number of tokens a language model can consider at once, encompassing both the input prompt and the generated output.