0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

Every Request Leaves a Receipt: Per-Request Cost Attribution for LLM Gateways

0
Posted at

Provider billing answers one question well: how much did the account spend? It answers almost no other question a team actually asks. Which feature generated this spend? Which team's experiment doubled Tuesday's usage? Which key belongs to the integration that retries every failure six times? The invoice knows the total. It does not know the story.

Per-request cost attribution is the practice of recording the story at the moment it happens. As documented in nRouter's LLM cost optimization framework, every request records its model, input and output tokens, cost, latency, retry count, and provider outcome — attributed to the team, key, and project that generated it.

Why the fields matter

Each field exists because a real cost review asks for it:

  • Model — because tier decisions (which tasks earn a premium model) can only be reviewed against actual model usage per task type.
  • Input/output tokens — because context growth and verbose outputs are separate problems with separate fixes, and the total hides which one you have.
  • Latency — because cost reviewed without latency invites savings that users pay for in waiting.
  • Retries — because failure waste is invisible in a success-only ledger; a request that succeeded on the fifth attempt cost five attempts.
  • Provider outcome — because fallback activity has a cost signature of its own.
  • Team, key, project — because unattributed spend has no owner, and unowned spend is never optimized.

Exports and the cost-quality review

Records that live only in a dashboard get looked at during incidents. Spend exports — the framework documents CSV exports — are what let cost review become a routine: pull the period's records, group by team and model, and read cost next to quality outcomes. The teams that find their waste fastest are not the ones with the best dashboards; they are the ones whose data leaves the tool.

Attribution is the third phase, not an afterthought

The framework orders cost control as prevent, route, attribute — and the order is a dependency chain. Prevention (caching, pre-dispatch budgets) and routing (least expensive capable model) make decisions; attribution is how you find out whether the decisions were right. Without receipts, every cost review is folklore. With them, it is arithmetic.

Disclosure: I work with the nRouter team. Capabilities described are as documented on the linked page.

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?