Also known as: finetuning · LoRA · PEFT · supervised fine-tuning
Fine-tuning is the process of further training a pre-trained model on a smaller, task-specific dataset to adapt its behaviour, style, or domain knowledge without training from scratch.
Parameter-efficient methods such as LoRA have largely displaced full fine-tuning for most applications. Rather than updating every weight, they train a small set of additional parameters, cutting compute and storage requirements by orders of magnitude while retaining most of the benefit.
A frequently-made mistake is reaching for fine-tuning to inject knowledge. Fine-tuning is well suited to teaching format, tone, and task structure; for supplying facts, retrieval is usually cheaper, more current, and more verifiable.
Large Language Model
A large language model (LLM) is a neural network trained on vast quantities of text to predict likely continuations of a sequence, producing systems capable of generating coherent language, answering questions, and writing code.
Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) is a technique in which relevant documents are retrieved from an external knowledge source and supplied to a language model as context, so that its response is grounded in specific retrieved material rather than parametric memory alone.
Open-Weight Model
An open-weight model is one whose trained parameters are publicly downloadable, allowing anyone to run, inspect, fine-tune, and deploy it on their own infrastructure without calling an external API.
Knowledge Distillation
Knowledge distillation is a training technique in which a smaller student model learns to reproduce the behaviour of a larger teacher model, yielding a compact model that retains much of the teacher's capability.
RLHF
RLHF (Reinforcement Learning from Human Feedback) is a training technique that aligns a model's behaviour with human preferences by training a reward model on human comparisons of outputs, then optimising the language model against that reward signal.