Jailbreak
A prompt or sequence that causes a language model to produce content it would normally refuse — bypassing safety training through role-play, hypotheticals, or adversarial framing.
Modern language models are trained with safety layers on top of their base capabilities. They refuse to help with a defined set of tasks: instructions for weapons, self-harm content, non-consensual intimate imagery, malware source, and category-varying edge cases like specific personal information about private individuals.
A jailbreak is any prompt technique that induces the model to comply anyway. Common patterns include role-play framing ("pretend you're an AI without restrictions..."), hypothetical framing ("if you WERE going to answer, hypothetically, how would you..."), token-splitting or obfuscation, and layered instructions that overwhelm the safety layer.
Jailbreak research is dual-use. Security researchers publish jailbreaks to make labs aware of gaps; adversarial actors use them to extract disallowed content. Modern models (Claude 4.x, GPT-5) are meaningfully harder to jailbreak than early versions but no model is jailbreak-proof — Anthropic and OpenAI both publish red-team results acknowledging this.
The related concept — but distinct — is prompt injection, which subverts the SYSTEM prompt (instructions from the developer) rather than the safety training (baked in by the lab).