Blog
For platform teams
You own the gateway, the keys, the rate limits, and the pager. The question you actually get asked is which layer to standardize on. Here is the honest map, with measured latency and fee math.
Platform teams choose among four gateway types: aggregators (OpenRouter, 5.5% on credits), control planes (Portkey, $49/month), self-hosted (LiteLLM, free plus ops), and blended models (Pareto, at cost). The measured hop latency is +68ms median; the fee crossover sits near $900/month of spend.
The latency answer, so you can stop asking it
We measured the gateway detour: +68ms median time-to-first-token versus direct, with the gateway's p95 beating direct because provider variance exceeds the hop. The script is published; run it against any layer you evaluate. The real latency question is per-request routing overhead on hard tasks, which we document for ourselves (up to 3x on hardest reasoning, +0 agentic) on the model card.
The operational trade nobody prices
Self-hosting is free the way a puppy is free. Our LiteLLM breakdown does the honest math: at two engineer-hours a week, running your own gateway costs about $1,000/month in loaded time before the first incident, against $49 flat for a hosted control plane or a 5% fee that only beats flat below ~$900/month of spend. If your team already runs platform infrastructure with on-call, marginal ops cost approaches zero and self-hosting wins; if not, you are volunteering for a second job.
What a blended model changes for you
Every option above still leaves model selection as a config file you own and re-litigate at every launch. Pareto collapses the routing table into the model itself: one endpoint, OpenAI-compatible surface (migration is a base URL and key), several LLMs on every request synthesized into one answer, billed at cost, receipts published. Provider variance, the quiet failure mode of aggregator catalogs, becomes a measured, owned problem instead of yours: see silent quality regression for the evidence.
Standardization checklist
Whatever you pick: confirm the OpenAI-compatible surface (portability), confirm streaming and tool-calling support, load-test the hop yourself with our published script, price the fee structure at your actual spend in the fee calculator, and demand published evidence from anything that claims quality. Vendors who publish harnesses can be checked; vendors who publish adjectives cannot.
Questions we get from this seat
Which gateway type should a platform team default to?
Constraint-first: data-locality requirements force self-hosted or VPC deployment; governance requirements favor a control plane; a bill problem favors the layer that changes what tokens cost rather than feeing them. The four-type map with fees is in our gateway guide.
How do we avoid gateway lock-in?
Standardize on the OpenAI-compatible surface and keep routing config exportable. Every serious option speaks it, so switching stays a base-URL change; the lock-in risk is accumulated balances and bespoke routing rules, not code.
What SLOs make sense for an LLM gateway?
Availability and TTFT percentiles per route, plus a quality canary: a fixed eval set run on schedule to catch silent regressions. Latency SLOs without quality monitoring miss the failure mode that actually hurts.