Also known as: on-device AI · local inference
Edge AI is the practice of running AI models directly on end-user devices such as phones and laptops rather than on remote servers, keeping input data local to the device.
The privacy argument is genuine: data that never leaves the device cannot be intercepted in transit, retained on a server, or repurposed for training. Latency and offline capability are additional practical benefits.
Two caveats are frequently glossed over. Some features marketed as on-device silently fall back to cloud processing under constraints such as low battery or unsupported languages, and data kept local is still retained locally — reachable by an attacker with device access or a device-targeted legal request.
Inference
Inference is the process of running a trained model to produce output for a given input — as distinct from training, which is the process of creating the model in the first place.
Quantization
Quantization reduces the numerical precision of a model's weights — for example from 16-bit floating point to 8-bit or 4-bit integers — shrinking memory requirements and speeding inference at some cost to output quality.
On-Device Processing
On-device processing means performing computation locally on a user's own hardware rather than transmitting data to a remote server, so that the raw input never leaves the device.
Differential Privacy
Differential privacy is a mathematical framework that adds carefully calibrated statistical noise to data or query results, providing a provable guarantee that the presence or absence of any single individual has negligible effect on the output.