Also known as: vector embedding · text embedding · semantic vector
An embedding is a numerical vector representation of text, images, or other data, positioned in a high-dimensional space such that semantically similar items sit close together.
Embeddings enable similarity search on meaning rather than exact wording, so a query about 'car maintenance' can retrieve a document about 'automobile servicing' despite sharing no keywords.
The important limitation is that embedding similarity is not relevance. Two passages can be semantically adjacent while only one actually answers the question, which is why production retrieval systems typically combine embeddings with keyword search and a re-ranking pass rather than relying on vector similarity alone.
Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) is a technique in which relevant documents are retrieved from an external knowledge source and supplied to a language model as context, so that its response is grounded in specific retrieved material rather than parametric memory alone.
Vector Database
A vector database is a data store optimised for indexing and querying high-dimensional embedding vectors, typically using approximate nearest neighbour algorithms to find the most semantically similar items quickly.
Semantic Search
Semantic search retrieves results based on meaning rather than keyword matching, using embeddings to find documents conceptually related to a query even when they share no literal terms.
Transformer
The transformer is a neural network architecture, introduced in 2017, that uses self-attention to weigh the relevance of every input token against every other, and which underpins essentially all modern large language models.