Also known as: AI jailbreak · safety bypass
A jailbreak is a prompt crafted to bypass an AI model's safety training and elicit output the model was trained to refuse, typically through roleplay framing, hypothetical scenarios, or obfuscated instructions.
Jailbreaks exploit the tension between instruction-following and refusal training. Because models are trained to be helpful and to follow context, framing a request as fiction, research, or a nested scenario can shift the balance toward compliance.
They are distinct from prompt injection: a jailbreak is the user attacking the model's own guidelines, whereas prompt injection is a third party attacking an application through content the model processes. Defences differ accordingly.
Prompt Injection
Prompt injection is an attack in which malicious instructions are embedded in content an AI system processes — a web page, document, or email — causing the model to follow the attacker's instructions instead of the application's.
AI Alignment
AI alignment is the field concerned with ensuring AI systems reliably pursue the goals their operators actually intend, rather than the literal specification given or some proxy the system learned during training.
RLHF
RLHF (Reinforcement Learning from Human Feedback) is a training technique that aligns a model's behaviour with human preferences by training a reward model on human comparisons of outputs, then optimising the language model against that reward signal.
AI Red Teaming
AI red teaming is the practice of deliberately probing a model or AI-powered application for harmful, unsafe, or policy-violating behaviour before and after deployment.