Inferbase vs NotDiamond
NotDiamond recommends a model; you run the inference. Inferbase routes and serves through one API: one decision, one bill.
Who runs the model
Both pick a model per request. The difference is who runs it.
Recommends the model
You get a recommendation. Running it, with your keys, providers, and fallback, is on you.
Routes and serves
One call picks the model and serves it, with a record of what ran and why.
Deciding, or delivering
A recommendation you run yourself, versus routing and serving as one system.
NotDiamond is a strong, eval-trained router, and it lets you train a custom router on your own data and even your own fine-tuned models. The line between it and Inferbase is execution.
- It recommends; you run it. By default NotDiamond returns a model choice and stops, so the providers, keys, and fallback stay yours to operate.
- Recommend-only is private. In that mode it never sees your outputs, a real benefit if you run everything yourself.
- Inferbase keeps decision and delivery together. One OpenAI-compatible call picks the model and serves it, so there is one bill and one audit record per request.
If you want a routing brain to drop into infrastructure you already run, NotDiamond fits. If you want routing and serving as one managed system, that is where Inferbase is built to win.
Side by side
Where the two line up and where they diverge, including what NotDiamond does well.
| Inferbase | NotDiamond | |
|---|---|---|
| What it does | Routes each request and serves the model | Recommends the best model per request |
| Inference execution | Included, managed serverless | Yours, you call the provider (or self-host their proxy) |
| Integration | One OpenAI-compatible API | SDK or OpenAI-compatible proxy; bring your provider keys |
| Per-request audit | One record: decision, model, tokens, cost, latency | Returns the chosen model and a session id; usage lives in your calls |
| Routing basis | First-party, benchmark-grounded | Eval-trained preference router |
| Customization | Custom model pools you define from the catalog | Train your own router on your evals and fine-tuned models |
| Optimize for | Quality, cost, or latency | Quality by default, tunable cost and latency tradeoff |
| Fallback and reliability | Handled by the platform | Yours, unless you run their proxy |
| Pricing | Free to start | Free Early Access; per-million-token routing fee; Enterprise custom |
Reflects publicly documented behavior as of June 2026. NotDiamond changes quickly, check their docs for the latest.
Where each one fits
A routing brain to assemble, or a platform that routes and serves. Pick by what you want to own.
NotDiamond is the better fit when
- You want to train a custom router on your own evals, including your fine-tuned models
- You want recommend-only routing that never sees your model outputs
- You want to keep your existing providers and gateway and add a routing brain
- You are routing inside your own agent or coding harness
Inferbase is the better fit when
- You want routing and execution in one OpenAI-compatible API, with no provider keys to wire up
- You want one per-request record tying the decision to the served tokens, cost, and latency
- You want managed serverless inference included, not just a recommendation
- You want to start in minutes without building an eval harness first
Frequently asked questions
Straight answers on how Inferbase and NotDiamond differ, and when each one is the better choice.
An LLM router decides which model should answer a request, per request, rather than leaving one model hard-wired into the application. It reads the prompt, works out the kind of task and how demanding it is, and picks from the models that are good enough for it on the objective you care about. Some routers only return the recommendation and leave the call to you; some, like Inferbase, also serve the request and keep the decision on record.
A router decides which model should answer; a gateway executes the request and governs it with keys, limits, budgets, logs and failover. Some routers only recommend, leaving you to call the provider; some gateways only proxy to the model you named. Inferbase runs the router inside the gateway, so the decision is served rather than recommended and its record sits beside the usage receipt.
That the same call which chooses the model also runs it. You send model="auto" to one OpenAI-compatible endpoint; the router classifies the request, picks the model that clears the bar for the task on your objective, the gateway serves it with two fallbacks ready, and the response comes back with its decision on the record. There is no second system to operate for the inference and no second bill for it.
When a workload mixes tasks and difficulties, which most production traffic does. We publish no savings percentage: one was withdrawn in September 2026 because the baseline price it was measured against is no longer one we can substantiate, and a ratio is only as good as its denominator. What routing does is send each request to the cheapest model that clears the quality bar for it, so the gain is the gap between what you pay now and what the right-sized model costs. A workload that is all one hard task gains less; the routing page shows the run and its caveats, and Routing Preview measures it on your own traffic.
No. NotDiamond recommends which model to call; by default you make the inference call yourself with your own provider keys, so it never sees your outputs. It also ships an optional, self-hostable OpenAI-compatible proxy, but even then the model runs on the underlying providers, not on NotDiamond. Inferbase routes and serves: the chosen model runs for you through one API, with no provider keys to wire up.
NotDiamond is an eval-trained preference router, and it lets you train a custom router on your own data and even your own fine-tuned models, which is a genuine strength if you have an evaluation harness. Inferbase’s routing is first-party and benchmark-grounded with an objective you set, and it comes with execution and a per-request audit trail. The honest framing is a tradeoff: a customizable routing brain you assemble, versus an end-to-end platform that routes and serves.
Yes. OpenRouter’s Auto Router is NotDiamond-powered, which is a fair signal that NotDiamond’s routing is good. Inferbase competes at that same routing layer directly, and adds the serving layer, so you get the decision and the inference from one place instead of assembling them.
In its recommend-only mode, NotDiamond returns a model choice without seeing your outputs and keeps your keys client-side, which is a real benefit if you run everything yourself. Inferbase serves the model, so it processes the request; the tradeoff is that you get routing, execution, and a single audit trail in one place rather than stitching them together.
Both offer OpenAI-compatible access. With Inferbase you point the base URL and key at us, set model="auto", and you are done, there are no provider keys to manage and no routing SDK to wire into your code.
On evidence, then on your objective. Each request is classified for its task family and complexity; the catalog is narrowed to the models that clear a quality bar for that task on benchmark evidence and that fit the request; the survivors are ranked on the objective you set (cost, balanced, quality or latency). A model with no measurement for the task is treated as unknown and does not qualify on guesswork.
Yes. Every routed request writes a decision record: the task the classifier saw, its complexity, the objective, how many models qualified, which won and how clearly, and each fallback attempt if any. When streaming it is the first event on the stream; afterwards it is retrievable by request id from the API and the dashboard.
One line. Point an existing OpenAI SDK at the Inferbase base URL and send model="auto"; chat completions, streaming, tool calls and structured output pass through unchanged, and Anthropic-shaped clients are served on the Messages endpoint. Name a model instead of auto and it is served as named, so an application can pin where it must and route where it can.
Free during beta. The published plan meters routing decisions at $1 per 1,000 with the first 5,000 each month free (a pinned model is never charged a routing fee), and managed inference at the host's token price plus a flat 6%. Connect your own provider keys and the tokens are billed by your provider instead. New accounts start with $5 of credit.
Put intelligence in the middle.
One gateway in front of every model, with your policies applied and every decision on record. Start with $5 of credit and 5,000 routing decisions a month, no card required.