01Cost per outcome, not per token
Tokens are an input; the number that matters is cost per resolved conversation or per processed document. That figure includes retries, failed attempts and the human minutes spent cleaning up.
Once you measure per outcome, comparisons become honest: a more expensive model that halves escalations is usually cheaper.
02Context is the silent budget
Stuffing the whole knowledge base into every request feels safe and quietly multiplies your bill. Retrieve narrowly, summarise long histories and cache what repeats.
Route by difficulty. Most requests are simple enough for a smaller model; reserve the expensive one for the cases that genuinely need it.
03Guardrails on spend, not just on content
Per-tenant and per-day ceilings, alerting on anomalies, and a hard cut-off prevent one bad loop from producing a five-figure surprise.
Log cost per request from the first day. Adding it later means you cannot explain the trend that already happened.