Route before you spend

Classify requests by complexity and sensitivity. Simple extraction can use smaller models or rules; only escalate to the largest model when metrics justify it. Routing policy is code—and should be versioned like code.

Instrument routing decisions: how often each path fires, and what quality delta you get from escalation. Without that, you cannot defend a more expensive tier.

Cache what repeats

Embed caches, retrieval bundles, and summarisation of stable corpora with explicit TTLs and invalidation. Do not cache personalised answers next to shared facts without segmenting keys.

Watch for cache poisoning and stale permissions: invalidate on auth changes and on document updates that affect retrieved content.

Budgets and brakes

Per-workflow and per-tenant caps, with graceful degradation: shorter context, cheaper model, or human handoff when limits approach. Visibility in Monitor ties spend to business events, not just dashboards for finance.

Chargeback and culture

Make cost visible to product owners early—weekly burn by feature, not a monthly surprise. When teams see spend next to quality metrics, trade-offs become technical conversations instead of blame cycles.