← AI Security & Ops theorytheory 0/14 · 0%

level 1

Curious · 0 XP

streak

perfect

badges

100 XP to Apprentice

Operations · medium

8. Cost governance and rate limiting

Budget per user, per feature, and per tenant — with hard stops.

1 min read2 questionsup to 50 XPtopic 8/14

Read in another language

AI translates the explanation and quiz text while preserving code and answer scoring.

Track spend where it is incurred and enforce ceilings before the call, not in a monthly invoice review.

sql
create table ai_usage (
  id bigserial primary key,
  user_id uuid not null,
  feature text not null,
  model text not null,
  prompt_tokens int not null,
  completion_tokens int not null,
  cost_usd numeric(10,6) not null,
  created_at timestamptz not null default now()
);
create index ai_usage_user_day on ai_usage (user_id, created_at desc);

ts
const spent = await dailySpend(userId);
if (spent > planCap(user.plan)) throw new QuotaError("daily AI limit reached");

Gateway error semantics matter here: 429 is transient — back off honouring `Retry-After`; 402 and 403 are terminal — surface the message and stop, and for background jobs persist a paused state so every scheduled run checks it before doing work. Never auto-retry a 402 or 403.

Cheapest real savings, in order: prompt caching on a stable prefix, model routing by task difficulty, shorter retrieved context, and deduplicating identical requests.

Exercise: write usage rows for every call, build a per-feature daily cost view, and add a hard cap that returns a clear message instead of a stack trace.

How to study this topic

1. Read the explanation once end to end, skipping the code.
2. Re-read the code snippets and say out loud what each line does.
3. Answer the quiz without scrolling back — that is the real check.
4. Read every explanation, even for the answers you got right.

Scoring

· Up to 30 XP from quiz accuracy.
· +15 XP for a perfect first attempt.
· +5 XP while a 3-day streak is running.
· 75% or higher marks the topic complete.

Check your understanding

1. How should a 429 from the AI gateway be handled?
Back off, honouring Retry-After, with a capped number of attemptsRetry immediately in a loopTreat it as terminalSwitch providers instantly
2. How should a 402 (credits exhausted) be handled in a background job?
Persist a paused state and stop all automated AI workRetry every minuteDowngrade the model and continueIgnore and log

Badges

First light

Pass your first topic

Sharp shooter

3 perfect first-try quizzes

On a roll

Study 3 days in a row

Week warrior

Study 7 days in a row

Grinder

Earn 500 XP

Deep diver

Earn 1500 XP