Budget Checks That Concurrency Cannot Beat

A budget check looks like the simplest code in a gateway: read how much has been spent, compare it with the limit, allow or refuse. It stops being simple the moment traffic is parallel. An agent framework can fire hundreds of requests at once, and if each one reads the same "spent so far" figure, each one concludes there is room — and the limit is honoured by every check and breached by their sum.

This article describes how nRouter Budgets structure the check so that concurrency cannot defeat it, and the refusal semantics that make a budget enforceable by machines rather than by meetings.

The check is a sum of three numbers, taken atomically

Every evaluation adds three quantities in a single atomic step:

1. Settled spend in the current window.
2. Reservations held by requests currently in flight.
3. The upfront reservation the incoming request needs.

If the total fits the budget, the request proceeds and its credits are reserved as part of the same step. If it does not fit, nothing is reserved and nothing is charged: the model never runs. The reservation is the important part. Without it, in-flight work is invisible to the check, and "spent" is always an underestimate at exactly the moments — bursts, fan-outs, retry storms — when the limit matters most.

Refusals are part of the API contract

A budget that fires has to tell the caller something it can act on. Three behaviours cover the space:

• Block returns HTTP 402 with a budget_exceeded error code, and the error names the budget that fired. Nothing was reserved, so there is no cleanup.
• Throttle returns HTTP 429, and pointedly does not queue the request or compute a Retry-After. A budget throttle is a wall, not congestion; a computed retry time would teach the client that waiting works, and it does not.
• Warn lets the request run and marks the response with a budget-warning header. It can also be attached to a soft threshold below the limit, so a caller learns about pressure while there is still headroom to react.

The common thread: the signal arrives in-band, on the response, in a form code can branch on — not in a dashboard a human might check tomorrow.

Limits compose, and the parent binds

Budgets exist for the organisation and for teams, administrators can set them for individual users, and virtual keys carry separate spend limits of their own. Every applicable budget is checked on every request. The composition rules are deliberately conservative: a team budget cannot be set higher than the organisation budget for a matching or longer period, and when a key limit sits above its parent budget, the parent simply acts first — the key limit is never reached, and the configuration carries an advisory saying so.

Windows, clocks, and failure

Reset periods are daily, weekly anchored to Monday, monthly, a fiscal year starting in a configured month, or lifetime. Window boundaries are computed on the database clock, which removes the classic drift where the gateway and the reporting dashboard disagree about when a window rolled over. Finally, if a budget record is corrupt or cannot be read, evaluation fails closed and the request is refused. A spend control that opens when its data breaks is a spend suggestion.

Takeaway

Treat the budget as a concurrency problem and an API-design problem, not a reporting feature: atomic checks with reservations, explicit refusal semantics, conservative composition, one clock, and fail-closed behaviour. Budgets built that way do not need a human watching them — which is the point of a budget.