Stop Retrying Budget 429s: The Ceiling Contract for LLM Gateways

A 429 that is not a rate limit

Retry logic encodes an assumption: the failure is transient. Backoff works because a rate limit is a statement about time — too many requests in a window — and time heals it. A budget refusal breaks the assumption. When an LLM gateway key crosses its budget ceiling, the next request is refused with a 429 and the code key_budget_exceeded. Nothing about waiting changes that state; the ceiling clears when its window resets or when a person raises the limit. A client that retries it is not being resilient. It is polling a decision.

The companion shape is a 402 with budget_exceeded, returned when an organization, team, or member ceiling fires instead of a key ceiling. Two statuses, two codes, one rule: branch on the code field, never on the status alone.

The model underneath

A budget is an amount, a reset window, and a scope. Windows: daily (midnight UTC), weekly (Monday, midnight UTC), monthly (the 1st), or total — a lifetime cap that never resets. Scopes stack: organization, team, member, and the individual key each carry a ceiling, all applying at once, and a request must clear every ceiling on its path. The check runs before the provider call, so a refused request is never a billed one. Tenant ceilings are owner-and-admin controls — the member whose ceiling fired cannot raise it, by design — and recovery is always deliberate, because an automatic escalation would be a ceiling that does not exist.

Handling it in code

Treat the two budget codes as non-retryable terminal states, distinct from a bare 429:

1. Parse the error body's code before deciding anything.
2. key_budget_exceeded (429): stop this key for the window; fail over or surface, never back off.
3. budget_exceeded (402): escalate to the scope's owner; the ceiling is organizational, not local.
4. A 429 with no budget code: a genuine rate limit — backoff is correct there.

On success paths, read x-nr-request-cost off every response and sum it by scope; where the header is absent, the call is unpriced and its cost is unknown — never default it to zero in your own accounting, because the forecast you build from those sums is only as honest as their gaps.

Why ceilings at all

Alerts and dashboards observe spend; a ceiling is the only control that can refuse it, and that refusal is what makes a forecast's worst case a chosen number instead of an extrapolation. Set ceilings from a measured window — p95 plus headroom, never the mean — and keep the organization ceiling above the sum of the team ceilings so an unplanned event has bounded room to live in. The full guide: LLM budget ceilings