← AI Agents & MCP theorytheory 0/14 · 0%

level 1

Curious · 0 XP

streak

perfect

badges

100 XP to Apprentice

Performance · hard

13. Cost and latency engineering

Agents multiply token spend by their step count — attack both factors.

1 min read2 questionsup to 50 XPtopic 13/14

Read in another language

AI translates the explanation and quiz text while preserving code and answer scoring.

Cost ≈ steps × context size × price. Levers, in order of payoff:

1. Cut steps — better tool descriptions and a planning step remove whole iterations. 2. Cut context — summarise old turns; truncate tool output; never resend a full document. 3. Route models — strong model for planning and final answers, cheap model for extraction and routing. 4. Cache — prompt caching on the stable system+tools prefix; result caching on pure tools. 5. Parallelise — run independent tool calls concurrently.

ts
const [orders, gas] = await Promise.all([tools.run("get_orders", a), tools.run("get_gas", b)]);

Keep the system prompt and tool schemas byte-stable across requests so provider prompt caching hits; a timestamp injected at the top of the prompt silently destroys every cache hit.

Track cost per successful task, not per request — an agent that costs twice as much but succeeds first time is cheaper.

Exercise: instrument every run with steps, prompt tokens, completion tokens, and cost; plot cost per successful task before and after adding tool-result truncation.

How to study this topic

1. Read the explanation once end to end, skipping the code.
2. Re-read the code snippets and say out loud what each line does.
3. Answer the quiz without scrolling back — that is the real check.
4. Read every explanation, even for the answers you got right.

Scoring

· Up to 30 XP from quiz accuracy.
· +15 XP for a perfect first attempt.
· +5 XP while a 3-day streak is running.
· 75% or higher marks the topic complete.

Check your understanding

1. Why keep the system prompt and tool schemas byte-stable?
Provider prompt caching only hits on an identical prefixModels reject changed promptsIt reduces step countIt improves tool recall
2. Best single metric for agent efficiency?
Cost per successfully completed taskTokens per requestRequests per secondAverage message length

Badges

First light

Pass your first topic

Sharp shooter

3 perfect first-try quizzes

On a roll

Study 3 days in a row

Week warrior

Study 7 days in a row

Grinder

Earn 500 XP

Deep diver

Earn 1500 XP