LiteLLM vs a hosted AI gateway: when to run your own proxy
Ayo Adeniran · Oct 18, 2026 · 5 min read
Ask how to call a hundred models through one API without sending your traffic through someone else, and the answer is usually LiteLLM. It is open source under the MIT licence, it speaks the OpenAI format, and it comes in two forms: a Python SDK you call from code, and a proxy server you run as your own AI gateway.
The real question is not whether LiteLLM is good. It is whether you want to operate a gateway, or pay someone else to. This post lays out what each path gives you and costs you.
What LiteLLM gives you
LiteLLM's open-source proxy covers most of what teams build a gateway for:
- 100+ providers behind one OpenAI-format API
- Virtual keys, users and teams, with budgets and rate limits per key, team and user
- Spend tracking, request and response logging, and Prometheus metrics
- Fallbacks and load balancing across deployments
- Callbacks into observability tools such as Langfuse and MLflow
Enterprise adds SSO and SCIM, OIDC and JWT auth, audit logs, secret managers with key rotation, org and team admins, a multi-region control plane, and 24/7 support with SLAs. LiteLLM prices it "by your annual gateway request capacity, deployment architecture, and support needs — never per token." The open-source tier is "$0 · Free forever · self-hosted", and the Enterprise page lists self-hosted and air-gapped deployment.
To use virtual keys you also run a Postgres database and set a master key, the admin key that creates the others.
What running it yourself means
Self-hosting is a trade, not a free lunch. With LiteLLM you get:
- Your traffic stays in your network. Requests go from your proxy to the providers you hold contracts with, and nowhere else.
- No per-token fee to a gateway. You pay providers directly.
- The code is yours to read and change.
And you take on:
- Running it: the proxy, its Postgres database, scaling, upgrades and monitoring.
- Every provider relationship: keys, contracts, rate limits and invoices, one per provider.
- The gateway's own uptime. When your proxy is down, every model is down, however healthy the providers are.
For a platform team that already runs Kubernetes and Postgres, that is ordinary work. For a product team of five, it is a second product.
The options side by side
| LiteLLM (open source) | LiteLLM Enterprise | OpenRouter | Vercel AI Gateway | Momor Gateway | |
|---|---|---|---|---|---|
| Who runs it | You | You | OpenRouter | Vercel | Momor |
| Price beyond tokens | Nothing | By request capacity and support | 5.5% or 8% on credit purchases | No markup; payment processing fees may apply | 5.5% on credit purchases |
| Provider keys | All yours | All yours | Theirs, or yours: first $25,000 a month free, then 5% | Theirs, or yours at no fee | Yours, plus Momor's for providers it resells; 5% on yours |
| Stands out for | Control | Governance, SSO, audit logs | 600+ models from 80+ providers | AI SDK, budgets, uptime data | Failover, data rules, workflows |
When LiteLLM is the right call
- Data cannot leave your network, or the deployment must be air-gapped.
- You already operate the infrastructure, and one more service is cheap for you.
- You want to change the gateway itself, not just configure it.
- You hold contracts with every provider you use and want nothing in between.
When a hosted gateway is the right call
- You want one bill instead of ten. OpenRouter and Vercel let you buy model access from many providers through one balance.
- You do not want to be paged for the gateway.
- You want routing and provider health handled for you. OpenRouter's default routing weighs price, throughput, latency, rate limits and reliability; Vercel publishes measured uptime per provider.
Where Momor fits
Momor Gateway is a hosted gateway built around the thing self-hosters care about most: their own provider contracts.
Your keys come first, by rule. Save several keys per provider. Prioritized keys are tried before Momor's keys, fallback keys after, and each prioritized key decides whether Momor's keys may ever stand in for it, never for the models it covers, or never for anything on its provider. Keys can be limited to certain models, API keys or members. Adding or replacing a key needs the owner to have confirmed it is them in the last five minutes, and every key is tested against the provider before it is saved.
Data rules enforced in the router. Turn on zero data retention for one call or for the whole organization: calls then run only on keys marked as covered by a zero-data-retention agreement, and file processing is refused. Call records keep metadata only, never prompts or responses, and are deleted after 90 days. OpenAI calls are sent with store: false.
Failover you would otherwise write yourself. A provider that fails before answering is stepped over. The conversation, with its reasoning and tool calls, is rebuilt in the next provider's format, and tool-call ids are rewritten into the form each provider takes, the same way every turn. A stream that breaks mid-answer is never stitched to another answer. Each account has its own circuit breakers, so one customer's throttling does not trip another's.
Same SDK. The OpenAI SDK works with a new base URL and key; the request passes through as you wrote it, translated only for Anthropic, Gemini and Vertex.
What to weigh against LiteLLM: Momor runs the gateway, so your requests pass through Momor on their way to the provider. The catalog is 12 providers rather than 100+. And each call on your own key costs 5% of its list price, where self-hosted LiteLLM costs you nothing per call.
If you would rather not run a proxy but still want your contracts to decide where calls go, start with the Gateway quickstart and the bring your own keys guide.