Blog
LLM gateway
Part of the Unbiased glossary: the vocabulary of LLM pricing and infrastructure, defined with receipts instead of adjectives.
An LLM gateway is a layer between applications and model providers that exposes one API surface, usually OpenAI-compatible, for many models. The four 2026 types: aggregators (OpenRouter), control planes (Portkey), self-hosted gateways (LiteLLM), and blended models (Pareto), differing in fees, operations, and who picks the model per request.
Why it matters
Provider lock-in is an integration problem before it is a pricing one. Gateways make model choice reversible, which changes negotiating position, failover options, and how quickly you can adopt whatever ships next month.
Measured: what the extra hop costs
We measured the gateway detour directly: twenty streamed requests direct to a provider versus twenty through a full routing gateway. The hop added +68ms at the median time-to-first-token, and the gateway's p95 beat direct (0.821s vs 0.925s), because provider-side variance exceeds the detour. Method and a runnable script in the gateway comparison; the full type-by-type guide is here.
Common questions
Do I need a gateway?
If you call one provider at prototype volume, no. The moment you want failover, comparison, or pricing leverage, some gateway layer pays for itself; which type depends on whether your constraint is data locality, governance, or the bill.
What does a gateway cost?
By type: 5.5% on credits (OpenRouter), 5% token markup (Requesty), $49/month flat (Portkey), $0 plus ops (LiteLLM), or at-cost billing (Pareto). Flat and percentage fees cross over just under $900/month of spend.
Do gateways degrade quality?
The hop does not, measurably. Provider variance on aggregator catalogs does: the same model served by different providers returns different quality, which is a selection-layer problem, not a network one.