Platform teams choose among four gateway types: aggregators (OpenRouter, 5.5% on credits), control planes (Portkey, $49/month), self-hosted (LiteLLM, free plus ops), and blended models (Pareto, at cost). The measured hop latency is +68ms median; the fee crossover sits near $900/month of spend.
The latency answer, so you can stop asking it
We measured the gateway detour: +68ms median time-to-first-token versus direct, with the gateway's p95 beating direct because provider variance exceeds the hop. The script is published; run it against any layer you evaluate. The real latency question is per-request routing overhead on hard tasks, which we document for ourselves (up to 3x on hardest reasoning, +0 agentic) on the model card.
Price the hop against your latency budget
Measured 2026-07-28: 20 streamed runs each way, same prompt, same hour, time-to-first-token. Direct 0.588s / gateway 0.656s at the median; 0.925s / 0.821s at p95. Runnable script in the gateway comparison.
The operational trade nobody prices
Self-hosting is free the way a puppy is free. Our LiteLLM breakdown does the honest math: at two engineer-hours a week, running your own gateway costs about $1,000/month in loaded time before the first incident, against $49 flat for a hosted control plane or a 5% fee that only beats flat below ~$900/month of spend. If your team already runs platform infrastructure with on-call, marginal ops cost approaches zero and self-hosting wins; if not, you are volunteering for a second job.
Self-hosting priced honestly, against every alternative
Ops cost = hours × loaded rate × 4.33 weeks. Fee structures verified 2026-08-18. Incidents, upgrades, and the second job of being the routing owner are not in the math; they are the point of the sentence above.
What a blended model changes for you
Every option above still leaves model selection as a config file you own and re-litigate at every launch. Pareto collapses the routing table into the model itself: one endpoint, OpenAI-compatible surface (migration is a base URL and key), several LLMs on every request synthesized into one answer, billed at cost, receipts published. Provider variance, the quiet failure mode of aggregator catalogs, becomes a measured, owned problem instead of yours: see silent quality regression for the evidence.
Standardization checklist
Whatever you pick: confirm the OpenAI-compatible surface (portability), confirm streaming and tool-calling support, load-test the hop yourself with our published script, price the fee structure at your actual spend in the fee calculator, and demand published evidence from anything that claims quality. Vendors who publish harnesses can be checked; vendors who publish adjectives cannot.
The standardization checklist, scored live
✓ = native fit, △ = achievable with work or an add-on, ✗ = wrong tool. Two of the four columns are not us, and for some requirement sets we lose; that is what makes the grid usable.
Questions we get from this seat
Which gateway type should a platform team default to?
Constraint-first: data-locality requirements force self-hosted or VPC deployment; governance requirements favor a control plane; a bill problem favors the layer that changes what tokens cost instead of adding a fee on top. The four-type map with fees is in our gateway guide.
How do we avoid gateway lock-in?
Standardize on the OpenAI-compatible surface and keep routing config exportable. Every serious option speaks it, so switching stays a base-URL change; the lock-in risk is accumulated balances and bespoke routing rules, not code.
What SLOs make sense for an LLM gateway?
Availability and TTFT percentiles per route, plus a quality canary: a fixed eval set run on schedule to catch silent regressions. Latency SLOs without quality monitoring miss the failure mode that actually hurts.
Compare any two models
List rates and dated competitor measurements: prices and measured bills. Pareto 26.9 measured task costs are not published. Verbosity: Edition 2.