Yashotantra Academy logo
← AI Security & Ops theorytheory 0/14 · 0%
level 1
Curious · 0 XP
streak
0d
perfect
0
badges
0

100 XP to Apprentice

Operations · hard

12. Incident response for AI systems

Know how to contain a bad prompt, a leak, or a provider outage.

1 min read2 questionsup to 50 XPtopic 12/14

Read in another language

AI translates the explanation and quiz text while preserving code and answer scoring.

AI incidents look different from ordinary outages: the system is up and answering, but wrongly or unsafely. Prepare specific playbooks.

- Bad output at scale — flag-flip to the previous prompt/model, then diagnose from traces. - Injection or exfiltration attempt — revoke the tool's credential, tighten egress, list every trace that touched the poisoned source, notify affected users if data left the system. - Provider outage or 5xx storm — fail over to a secondary model, degrade to a non-AI path (deterministic search, template reply, queue for later) rather than showing a spinner. - Cost spike — per-user and global circuit breakers trip automatically; investigate a retry loop first, since that is the usual cause.

ts
export async function withFallback(fn: () => Promise<string>) {
  try { return await fn(); }
  catch (e) {
    if (isTerminal(e)) return TEMPLATE_REPLY;          // 402/403: stop, do not retry
    return await secondaryModel().catch(() => TEMPLATE_REPLY);
  }
}

Kill switches must be verified in a drill. An untested feature flag is a hope, not a control.

Exercise: run a game day — disable the primary model in staging and confirm the fallback path, the alert, and the user-facing message all behave as documented.

How to study this topic

  1. 1. Read the explanation once end to end, skipping the code.
  2. 2. Re-read the code snippets and say out loud what each line does.
  3. 3. Answer the quiz without scrolling back — that is the real check.
  4. 4. Read every explanation, even for the answers you got right.

Scoring

  • · Up to 30 XP from quiz accuracy.
  • · +15 XP for a perfect first attempt.
  • · +5 XP while a 3-day streak is running.
  • · 75% or higher marks the topic complete.

Check your understanding

  1. 1. First move when quality degrades across production?

  2. 2. Correct degradation when the provider is down?

Badges

First light
Pass your first topic
Sharp shooter
3 perfect first-try quizzes
On a roll
Study 3 days in a row
Week warrior
Study 7 days in a row
Grinder
Earn 500 XP
Deep diver
Earn 1500 XP