4 types
aggregators, control planes, self-hosted gateways, and blended models. Different fees, different jobs
Blog
LLM gateways: the complete guide
One integration, many models: that is the promise of the gateway layer. This guide covers the four types, what each actually charges, the latency cost we measured, and the selection question nobody prices: who picks the model for each request.
+68ms
the median time-to-first-token we measured for a full routing hop; p95 was better through the gateway than direct
~$900/mo
the spend level where flat platform fees beat percentage fees. Below it, percentages are the cheap convenience
An LLM gateway is a layer between your application and model providers that gives you one API surface for many models. The four types in 2026: aggregators (OpenRouter, 5.5% fee on credits), control planes (Portkey, $49/month flat), self-hosted gateways (LiteLLM, free plus ops), and blended models (Pareto, billed at cost) that pick the model for you.
What a gateway actually does
Every gateway solves the same integration problem: your code speaks one API (almost always the OpenAI-compatible surface) and the gateway speaks to every provider. What differs is everything else: who pays for it and how, who operates it, who owns the data path, and, most consequentially, who decides which model serves each request. The categories below sort by those answers.
The four types, with real fees
| Type | Example | Fee model | Who picks the model | Full comparison |
|---|---|---|---|---|
| Aggregator | OpenRouter | 5.5% on credit top-ups | you, from a 400+ catalog | OpenRouter alternatives |
| Control plane | Portkey | $49/mo flat | you, with governance | Portkey alternatives |
| Self-hosted | LiteLLM | $0 + your ops time | you, in a config file | LiteLLM alternatives |
| Blended model | Pareto | models at cost | several LLMs per request | model card |
Requesty (5% token markup, caching included) sits between aggregator and control plane. Fees verified 2026-07-28 from public pricing pages.
What the fees cost at your scale
Percentage fees scale with spend; flat fees do not. The crossover lands just under $900 a month: below it OpenRouter's 5.5% is cheaper than Portkey's $49 flat, above it the percentage becomes a growing tax. At $40k/month, 5.5% is about $26,400 a year against $588 flat. Run your own number in the gateway fee calculator. And keep the fee in proportion: on our measured four-task workload, routing the work to the right models saved 14x what the fee cost. The fee is not the money; the model selection is the money.
The latency question, measured
The standing objection to any gateway is the extra hop. We measured it: twenty streamed requests direct to a provider versus twenty through a full routing gateway, same prompt, same hour. The gateway added +68ms at the median time-to-first-token, and its p95 was actually better than direct (0.821s vs 0.925s), because provider-side variance is larger than the hop. Method and a runnable script in the gateway comparison. Latency becomes real when heavy routing reasoning runs on hard tasks; we document ours (up to 3x on the hardest reasoning, +0 on agentic work) on the model card.
Provider variance: the quiet problem
On aggregators, the same open-weights model served by different providers returns measurably different quality; OpenRouter built telemetry over billions of tool calls and launched Exacto specifically because of it. "Same model, same price" on a catalog does not mean same output. This is an argument for receipts at the layer that picks the model, whoever that is.
How to choose
Start from your constraint, not the feature grid. Prototyping across many models: aggregator. Data cannot leave your infrastructure: self-hosted (or enterprise VPC deployment). Compliance and visibility are the requirement: control plane. You want the bill down without making model selection your standing job: a blended model that routes per request and shows receipts. The Stack Finder asks a few quick questions and gives you a defensible starting answer; the interactive version of this guide's decision logic is in the comparison post.
Gateway questions, answered
What is the difference between an LLM gateway and a router?
Gateway describes the integration layer: one API surface, many providers. Routing is a capability some gateways add: choosing which model serves a request. Aggregators leave routing to you; blended models like Pareto go further, running several LLMs on every request and synthesizing one answer, billed at cost.
Do gateways add latency?
A hop costs something; we measured +68ms median time-to-first-token for a full routing gateway versus a direct provider call, with p95 better through the gateway. Provider-side variance is bigger than the detour. Heavy per-request reasoning on hard tasks is the real latency cost to ask about.
Do gateways mark up token prices?
By type: aggregators mostly pass list prices through and fee the credits (OpenRouter, 5.5%); Requesty marks tokens up 5%; control planes charge flat platform fees with tokens billed direct; self-hosted is markup-free by construction; blended models change the basis entirely by billing at cost.
Which gateway is cheapest?
Below roughly $900/month of spend, percentage-fee aggregators; above it, a flat-fee control plane, with self-hosted cheapest of all if your ops time is genuinely spare. Cheapest fee is a different question from cheapest bill; the selection layer moves more money than the fee layer.
Can I switch gateways without rewriting my code?
Usually yes: the OpenAI-compatible surface is the category standard, so switching is a base-URL and key change. Gateway-specific routing parameters do not travel; budget an afternoon to re-express them.
The type that picks for you. $100 to verify.
Buy Pareto API credits