How Inferbase compares to other LLM routers
Routers recommend, aggregators resell routing, gateways follow rules you write, frameworks make you host it. Inferbase decides on evidence, serves the model and keeps the record, in one API.
| Inferbase | OpenRouter | LiteLLM | Portkey | NotDiamond | RouteLLM | |
|---|---|---|---|---|---|---|
| Picks the best model per request | First-party | Via NotDiamond add-on | Beta tiers you map by hand | No, rules you define | Yes | Strong vs weak only |
| Routes and serves in one API | Yes | Yes | Proxies via your providers | Proxies via your providers | No, you run it | No, self-hosted |
| Per-request decision audit | Yes | No | Logs and cost tracking | Deep logs and traces | Recommend-side | Build your own |
| Nothing to self-host or calibrate | Yes | Yes | No | Hosted, rules are yours | Yes | No |
| Model breadth | Curated catalog of open models | Hundreds of models | 100+ providers, your keys | 1,600+ models, your keys | Your chosen pool | Two models |
The LLM routing landscape
Most tools sit in one of a few buckets: aggregators, gateways, routing brains and frameworks. Inferbase is the one that both decides and delivers.
OpenRouter
AggregatorOne API to hundreds of models. Routing is an add-on, and its Auto Router is outsourced to NotDiamond.
LiteLLM
OSS gatewayA self-hosted gateway to 100+ providers. It balances hosts for the model you named; its beta auto-router adds complexity tiers, with the tier-to-model mappings still in your config.
Portkey
Hosted gatewayA control panel over your own providers: observability, guardrails, and routing rules that key on metadata you attach, never the prompt itself.
NotDiamond
Routing brainRecommends the best model per prompt, then leaves you to run the inference with your own providers.
RouteLLM
OSS frameworkA free, self-hosted strong-vs-weak router you install, calibrate, and operate yourself.
Inferbase
Routes and servesPicks the best model per request across a curated catalog on benchmark evidence and runs it, through one OpenAI-compatible API, with a decision you can audit. First-party, end to end.
Why route at all
No single model is the right choice for every request. Routing turns model selection from a standing decision into a per-request one.
The right model is not the same from one request to the next, and a single default pays for that twice: once in money on the easy work, once in quality on the hard work.
This page compares Inferbase against the routers, gateways and aggregators teams usually weigh, OpenRouter, LiteLLM, Portkey, NotDiamond and RouteLLM, on the criteria that distinguish them. If you are still weighing the categories themselves, start with the breakdown of gateways versus routers.
A frontier model is overkill for easy work.
Classification, short summaries, and simple Q&A do not need a top-tier model, so paying for one wastes money on the bulk of your traffic.
A small model falls short on hard work.
Complex reasoning and long-context analysis need a stronger model, so a lean default quietly loses quality where it matters most.
A single default leaves both on the table.
Routing picks a model per request instead, so spend and quality each track the difficulty of the work.
23 more prompts went to five other models, the priciest of them held back for the hardest generation work. 10 stayed on the baseline, where nothing cheaper cleared the quality bar.
Every rate above is what that model costs per million tokens on the route that served it. What a workload saves depends on what it was paying before, which is a question about your bill rather than ours.
How to choose an LLM router
Four questions that separate the categories, and where the real quality and cost gains hide.
Does it pick the model, or just the host?
Optimizing the provider for a model you already chose is not the same as choosing the right model per request, which is where the quality and cost gains are.
Does it run the inference, or just decide?
A router that only recommends leaves you to operate providers, keys, and fallback. Routing plus serving is one system instead of two.
Can you see why it chose?
Without a per-request decision trail, routing is a black box. An auditable decision matters for trust and for debugging what ran.
Managed, or yours to operate?
Self-hosted frameworks are free, but you own the server, the threshold calibration, and the upkeep as frontier models change.
How Inferbase approaches it
Model selection as a per-request decision, with execution and an audit trail in the same place.
Inferbase treats model selection as a per-request decision: each prompt is classified, narrowed to the models that clear the bar for its task, ranked on the objective you set, and routed to the winner with two fallbacks ready.
The same call that picks the model also serves it and records why. How LLM routing works →
It serves, not just decides.
Unlike a routing brain, Inferbase runs the chosen model: one endpoint, one bill, one record.
Routing is first-party.
Unlike an aggregator, selection is benchmark-grounded, not outsourced to a third-party router.
Nothing to operate.
Unlike a self-hosted framework, there is no router server to run or threshold to calibrate.
Inferbase Gateway
One OpenAI-compatible API, with failover, rate limits and budgets behind it.
- knowledge
Catalog
Curated creators, verified variants, benchmark evidence where it exists, live prices.
- decision
Routing
A prompt classified, a model chosen on your objective, the decision kept.
- policy
Guardrails
Which models may serve, guardrails at three doors, which tools an agent may call.
- record
Observability
Decision, verdicts, tokens, cost and latency per request; usage and an audit log.
Read the full comparison
Each one leads with the routing-philosophy difference, then a dated side-by-side table with its sources, and where each tool fits.
Routing is a NotDiamond-powered add-on.
Read the comparisonHosts balanced; tier maps are yours to write.
Read the comparisonRules key on metadata, not the prompt.
Read the comparisonRecommends a model; you run it.
Read the comparisonSelf-host and calibrate it yourself.
Read the comparisonFrequently asked questions about LLM routers
What a router, a gateway and an aggregator are, how to tell them apart, and how Inferbase behaves.
An LLM router decides which model should answer a request, per request, rather than leaving one model hard-wired into the application. It reads the prompt, works out the kind of task and how demanding it is, and picks from the models that are good enough for it on the objective you care about. Some routers only return the recommendation and leave the call to you; some, like Inferbase, also serve the request and keep the decision on record.
A router decides which model to use for each request. A gateway such as LiteLLM or Portkey unifies many providers behind one API, with the model choice staying in config or in rules you write. An aggregator such as OpenRouter is one hosted API to many models, where routing between models is an optional add-on. Inferbase is routing-first and also serves the model, so the decision and the inference come from one place.
Load balancing spreads identical requests across hosts of the same model so that none is overloaded; it does not look inside the request. LLM routing looks inside: two requests to the same endpoint can go to different models because they ask for different things. A gateway that balances hosts for the model you named is not choosing a model; a router is.
Fallback reacts to a failure: the first model errors or times out, so a second is tried. Routing decides up front which model should go first. Most gateways offer fallback; far fewer make the first decision on the prompt. Inferbase does both: every routed request carries a primary and up to two fallbacks in the order the router ranked them.
When a workload mixes tasks and difficulties, which most production traffic does. We publish no savings percentage: one was withdrawn in September 2026 because the baseline price it was measured against is no longer one we can substantiate, and a ratio is only as good as its denominator. What routing does is send each request to the cheapest model that clears the quality bar for it, so the gain is the gap between what you pay now and what the right-sized model costs. A workload that is all one hard task gains less; the routing page shows the run and its caveats, and Routing Preview measures it on your own traffic.
Four questions separate the categories. Does it pick the model, or only the host for a model you already chose? Does it run the inference, or only recommend? Can you see why it chose, per request? And is it managed, or yours to host and calibrate as models change? The quality and cost gains live in the first; the operating cost lives in the other three.
With a recommend-only router, a gateway or a self-hosted framework, yes: you bring your own providers and keys and run the inference yourself. With Inferbase, no. It routes and serves through one OpenAI-compatible API on a managed pool, and if you prefer, it routes and serves on your own provider keys instead.
Mostly not in the sense this page uses. OpenRouter is an aggregator: one API to hundreds of models, where you name the model and it picks a host for it. Its automatic model selection is an optional feature, and that feature is powered by NotDiamond. The full comparison page sets out what each one routes.
That the same call which chooses the model also runs it. You send model="auto" to one endpoint; the router classifies the request, picks the model that clears the bar for the task on your objective, the gateway serves it with two fallbacks ready, and the response comes back with its decision on the record. There is no second system to operate for the inference and no second bill for it.
On evidence, then on your objective. Each request is classified for its task family and complexity; the catalog is narrowed to the models that clear a quality bar for that task on benchmark evidence and that fit the request; the survivors are ranked on the objective you set (cost, balanced, quality or latency). A model with no measurement for the task is treated as unknown and does not qualify on guesswork.
Yes. Every routed request writes a decision record: the task the classifier saw, its complexity, the objective, how many models qualified, which won and how clearly, and each fallback attempt if any. When streaming it is the first event on the stream; afterwards it is retrievable by request id from the API and the dashboard.
They are OpenAI-compatible to varying degrees, so switching is mostly a base URL and a key change. Moving to Inferbase needs no provider keys and no routing SDK in your code; you set model="auto", or name a model to pin it. Moving away is the same change in reverse, since nothing of Inferbase lives in your code.
With a recommend-only router or a self-hosted framework, inference runs on your own providers with your own keys, so the routing layer may never see your outputs. An aggregator or a managed platform such as Inferbase processes the request in order to run the model; on Inferbase, prompt and response content is kept for 30 days by default for your own activity log and audit, and zero-retention can be set per user or organization.
Free during beta. The published plan meters routing decisions at $1 per 1,000 with the first 5,000 each month free (a pinned model is never charged a routing fee), and managed inference at the host's token price plus a flat 6%. Connect your own provider keys and the tokens are billed by your provider instead. New accounts start with $5 of credit.
Put intelligence in the middle.
One gateway in front of every model, with your policies applied and every decision on record. Start with $5 of credit and 5,000 routing decisions a month, no card required.