A demo needs a model and a prompt. A product needs four more layers:
- Security — injection defence, PII handling, output filtering, tool authorisation. - Observability — traces, token and cost accounting, quality scores, alerts. - Memory — durable user and session state outside the context window. - Automation — schedulers and workflow engines that run AI work reliably and idempotently.
ts
export async function aiCall(input: string, ctx: Ctx) {
const span = tracer.start("ai.generate", { userId: ctx.userId });
const screened = await screenInput(input); // security
if (!screened.ok) return span.blocked(screened.reason);
const res = await gateway.chat({ model: "openai/gpt-5.6-sol", messages: build(screened.text) });
span.end({ promptTokens: res.usage.prompt_tokens, completionTokens: res.usage.completion_tokens });
return redactPii(res.text); // output filtering
}Every AI request should pass through one wrapper like this. Scattered direct SDK calls make it impossible to add a guardrail or a cost metric later without touching every file.
Exercise: find every direct model call in a project and route them through a single `aiCall` helper. Then add cost logging in one place and confirm all features report it.