Every provider a team adds brings its own SDK, API key, error taxonomy, and invoice. Past the second one, the team is maintaining an adapter layer — an internal interface, a few implementations, and a translation function whose whole job is to make several APIs look like one API. This article covers moving the seam to the endpoint: keep the OpenAI SDK, change two arguments, and let the model be a string.
What changes, in code
The migration is the constructor. The rest of the call — messages, parameters, streaming — stays as written:
from openai import OpenAI
client = OpenAI(
base_url="https://api.nrouter.ai/v1",
api_key="sk-nrouter-...",
)
resp = client.chat.completions.create(
model="nrouter/auto", # or any concrete model id from the catalog
messages=[{"role": "user", "content": "Summarize this diff."}],
)
The gateway authenticates the key, resolves the model string, translates the request into whatever the target provider expects, calls it, and translates the response back into the protocol shape the SDK already understands. You hold no provider keys and no provider SDKs; the one key you hold is divisible into scoped, revocable keys per environment and per service.
The catalog is live, not a list you embed
The models behind your key are your catalog, live and per-environment. Query the models endpoint or the models page; what is returned is authoritative, and a model not in the catalog is not reachable no matter what the request names. Adding one to an application is a string. Retargeting a router alias is a dashboard change with no deploy. Hardcoding a model list into your codebase is how catalogs age badly.
Keep the receipts: raw responses and headers
Plumbing is not the whole story — breadth usually costs visibility, so read the headers from the first call. Using the SDK's raw-response wrapper preserves them: x-nr-model reports the model that actually served the call (a mismatch against what you sent is a failover to an equivalent model from another provider, made visible); x-nr-request-cost reports the settled cost of that call and is absent when the call is unpriced — absent means unknown, never zero; x-nr-cost-status says exact or unpriced; x-nr-request-id ties the call to logs and traces. One response shape, whichever provider served it.
The honest limits
No bring-your-own-key: a rate negotiated directly with a provider does not travel through this seam, because usage settles from the gateway's own accounts. The gateway is a hop — your availability now includes it, and its status page is public. Compatibility is the OpenAI protocol, not vendor-private extensions. A custom fallback chain fires on any delivery failure; it cannot be scoped to one failure type. And a string change adopts the plumbing, not the evaluation: judge a new model on cost per successful, quality-passing request before it takes traffic. The full guide, with the adapter-layer argument and the effort honestly scoped: One API key across LLM providers