Budget-Aware LLM Routing & Fallback

Built cost guardrails into the LLM layer — per-tenant budgets, model downgrades under load, response caching, and graceful multi-provider fallback.

Context: LLM costs can spiral and providers fail. Decision: route every call through a budget-aware layer that enforces per-company rate limits, downgrades to cheaper models when appropriate, caches by content hash, and degrades gracefully instead of erroring. Trade-off: some routing complexity for predictable cost and reliability.