The hardest part of an LLM incident is rarely the failure. It is the reconstruction: which provider served the request, which model actually answered, how long it took, whether it rerouted on the way — and what it cost. In a multi-provider setup without a gateway, those answers live in different dashboards, at different granularities, with different retention. Some of them live nowhere.
A gateway sees every request, so it can record every request uniformly. In nRouter's LLM gateway, each request leaves a record with the same fields:
- Provider — which upstream served the call (or was attempted).
- Model — the catalogue entry that answered.
- Latency — how long the request took, end to end.
- Outcome — success, failure, reroute, or blocked, with the reason where a guardrail stopped it.
- Cost — attributed to that request, from the same usage ledgers that enforce budgets.
Three questions this answers without an archaeology project:
- "What happened during the outage?" Filter by outcome and time: which requests rerouted through the fallback chain, where they landed, and how latency moved while they did.
- "Why did spend jump on Tuesday?" Attribute cost per request, per key, per model. A ledger-backed record separates a traffic increase from a retry loop from a routing change.
- "Did the guardrail actually run?" Blocked requests are logged with their reasons. The check is auditable from the same record stream as everything else, not from a separate tool someone has to remember to open.
The discipline is the uniformity. Logs that exist for one provider integration but not another, or for chat but not embeddings, recreate the original problem inside the observability tool itself. One path for every request means one record shape for every request — which is most of what "observability" has to mean at a gateway.
Disclosure: I work with the nRouter team. Every capability and limit above is stated on the linked page.