← AI Security & Ops theorytheory 0/14 · 0%

level 1

Curious · 0 XP

streak

perfect

badges

100 XP to Apprentice

Security · easy

3. Jailbreaks and refusal robustness

Layer independent checks instead of trusting the model to refuse.

1 min read2 questionsup to 50 XPtopic 3/14

Read in another language

AI translates the explanation and quiz text while preserving code and answer scoring.

Jailbreaks use role play, hypotheticals, encoding tricks, translation, and long-context distraction to get past alignment. Because they exploit the model's flexibility, the fix is layered:

1. Input classifier — cheap model or rules flag policy-violating intent before the main call. 2. Constrained interface — structured output and a fixed tool set shrink the attack surface. 3. Output classifier — screen the response before it reaches the user or a tool. 4. Rate limits and abuse signals per user, since jailbreaking is iterative.

ts
const verdict = await classify(input, ["allowed", "self_harm", "malware", "pii_request"]);
if (verdict !== "allowed") {
  await abuse.record(userId, verdict);
  return SAFE_REFUSAL[verdict];
}

Log every block with the input hash (not the raw text) so you can measure attack volume without building a corpus of harmful content. Keep refusals short and non-instructive — a refusal that explains exactly which filter fired is a tutorial for the attacker.

Exercise: assemble 20 red-team prompts, measure the block rate with and without the input classifier, and check the false-positive rate on 100 benign prompts.

How to study this topic

1. Read the explanation once end to end, skipping the code.
2. Re-read the code snippets and say out loud what each line does.
3. Answer the quiz without scrolling back — that is the real check.
4. Read every explanation, even for the answers you got right.

Scoring

· Up to 30 XP from quiz accuracy.
· +15 XP for a perfect first attempt.
· +5 XP while a 3-day streak is running.
· 75% or higher marks the topic complete.

Check your understanding

1. Why layer input and output classifiers around the model?
A single model-level refusal can always be argued aroundClassifiers are fasterIt removes the need for logsProviders require two calls
2. What must you also measure when adding a safety classifier?
The false-positive rate on benign trafficGPU utilisationIndex sizePrompt length

Badges

First light

Pass your first topic

Sharp shooter

3 perfect first-try quizzes

On a roll

Study 3 days in a row

Week warrior

Study 7 days in a row

Grinder

Earn 500 XP

Deep diver

Earn 1500 XP