0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

A 402 Is a Budget Answer, Not an Error: Branching Retries on the Refusal Code

0
Posted at

The demo-to-production gap is configuration

A working prototype answers none of production's questions: whose key is this, what stops it spending all weekend, what did that call cost, where is the log when it fails. Through an LLM gateway those four jobs stop being code and become configuration. This note covers the launch sequence — and the one detail most retry logic gets wrong.

Reserve before you send

Funding comes first because the accounting starts before the request does. nRouter reserves each call's estimated cost against the organization's balance before the request leaves the gateway; if the balance cannot cover it, the call returns 402 and no provider is contacted. A ceiling that holds before the spend exists is a different thing from an alert that fires after it.

One key per service

Create a scoped virtual key per (environment, service) pair: name, one team, per-key budget, allowed models and endpoints, an expiry. The full value is shown once. A single shared key makes attribution permanently impossible — one undifferentiated line item that no later logging can separate.

The base URL is the integration

The API is OpenAI-compatible, so the client already in your codebase takes a new base_url and nothing else changes. From the first call, read the headers: x-nr-request-id is always present — store it on success and failure — and x-nr-request-cost reports the settled cost when the call is priceable. Absent cost header means unknown, never zero.

Branch on the code, not the status

Attach the budget at the scope matching the risk, and let retry logic read the error code:

Scope Status Code
Organization / team / user budget 402 budget_exceeded
Per-key budget 429 key_budget_exceeded

A rate-limit 429 self-clears; a budget 429 does not clear until the window resets. Retrying a budget block with exponential backoff is the classic first-week bug — the client hammers a refusal that time cannot fix. Branch on the code field and the two cases separate cleanly.

Verify the ceiling fires

Before real traffic: attach a tiny total budget to a throwaway key, send two calls, confirm the second is refused with the budget code, delete the key. Then the second model is a string change, not a second integration.

The full five-step guide, with the complete verification sequence: Ship AI features faster

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?