Yashotantra Academy logo
← AI Agents & MCP theorytheory 0/14 · 0%
level 1
Curious · 0 XP
streak
0d
perfect
0
badges
0

100 XP to Apprentice

Projects · hard

14. Capstone: a production research agent

Assemble loop, MCP tools, memory, guardrails, approvals, tracing, and evals.

1 min read2 questionsup to 50 XPtopic 14/14

Read in another language

AI translates the explanation and quiz text while preserving code and answer scoring.

Build an agent that answers questions about your own documents and data, and can file a summary ticket.

Architecture:

1. Tools — an MCP filesystem/docs server for retrieval, a SQL tool for metrics, `create_ticket` behind an approval gate. 2. Loop — ReAct with a max of 10 steps and per-tool timeouts. 3. Memory — scratchpad compression at 12k tokens; long-term facts in a table with a vector column. 4. Guardrails — input injection screen, output PII redaction, allow-listed egress. 5. Observability — one trace per run with step spans, token counts, and cost; sample traces into a review queue. 6. Evals — 30 trajectory cases in CI with mocked tools; nightly staging run.

ts
export async function research(goal: string, userId: string) {
  const trace = startTrace({ userId, goal });
  try {
    const answer = await runAgent(goal, registry, 10);
    trace.ok({ answer });
    return answer;
  } catch (e) {
    trace.fail(e);
    return "I could not complete this safely. A human has been notified.";
  }
}

Ship criteria: every run traced, every irreversible action gated, eval suite green, and a documented per-task cost ceiling with an alert when it is exceeded.

Exercise: run the capstone against ten real questions from colleagues and log which failures were retrieval problems, which were tool-schema problems, and which were genuine reasoning errors. Fix in that order.

How to study this topic

  1. 1. Read the explanation once end to end, skipping the code.
  2. 2. Re-read the code snippets and say out loud what each line does.
  3. 3. Answer the quiz without scrolling back — that is the real check.
  4. 4. Read every explanation, even for the answers you got right.

Scoring

  • · Up to 30 XP from quiz accuracy.
  • · +15 XP for a perfect first attempt.
  • · +5 XP while a 3-day streak is running.
  • · 75% or higher marks the topic complete.

Check your understanding

  1. 1. Which ship criterion matters most for safe rollout?

  2. 2. In production triage, which failure class is usually largest?

Badges

First light
Pass your first topic
Sharp shooter
3 perfect first-try quizzes
On a roll
Study 3 days in a row
Week warrior
Study 7 days in a row
Grinder
Earn 500 XP
Deep diver
Earn 1500 XP