When model is auto, Inferbase classifies the request, keeps the models in the policy's pool that clear the evidence bar for that task and fit the request, ranks them on the objective, and serves the winner with fallbacks ready. The pool, objective, and scope come from the routing policy the key runs under: the policy attached to the key, else the project default, else the organization default, else the Inferbase default. Individual requests need no routing fields. See How routing works.
Per-request narrowing is done with the routing object on chat completions.
The routing event
On a streamed response the first event, before any content, is the decision.
data: {"object": "routing", "task": "coding", "complexity": "simple", "estimated": false, "decisiveness": 0.95, "forced_single": false, "model": "openai/gpt-oss-120b", "measured": true, "optimize_mode": "balanced", "routing_time_ms": 112}| Field | Meaning |
|---|---|
task | The task the request was classified as. |
complexity | How demanding it was judged to be. |
estimated | true when classification fell back to a heuristic estimate. |
model | The model chosen, as provider/model-id. |
measured | false when the pick has no benchmark evidence for the task. |
optimize_mode | The objective in force. |
decisiveness | The winner's margin over the runner-up, 0 to 1. null when only one model qualified. |
forced_single | true when only one model qualified, so a forced pick never reads as a decisive win. |
routing_time_ms | Time spent choosing, measured on this request. It scales with prompt length; read the value your own traffic returns. |
session | Whether the conversation stayed on its model, and why. |
cache_fit | Whether a prompt-cache preference was satisfied, and how the chosen provider caches. |
user_rules_applied | Which policy settings shaped the decision, and any accepted fields that were not forwarded. |
OpenAI SDKs skip events they do not recognize; read the raw stream to consume it in band.
Routing decisions
| Method and path | GET /routing-decisions/{request_id} |
| Purpose | The persisted decision for one routed request: classification, the eligible pool, the fallback chain, decisiveness, and the per-step eligibility trail. |
| Authentication | API key or session |
request_id is the id of the chat completion response. Log it beside your own records and fetch the decision whenever you need it.
Response, abridged
{
"request_id": "req-9f2c41d8a6b34e17",
"routing_mode": "balanced",
"classifier_task_family": "coding",
"classifier_complexity": 0.41,
"eligibility_pool_size": 12,
"final_disposition": "served",
"decisiveness": 0.73,
"forced_single": false,
"chain_attempted": ["<route id>", "<route id>", "<route id>"],
"route_displays": {"<route id>": "GPT OSS 120B"},
"filter_trail": [
{"step": "pool_loaded", "in": 20, "out": 20},
{"step": "context_fit", "in": 20, "out": 16},
{"step": "preset_eligibility", "in": 16, "out": 12, "preset": "standard"},
{"step": "chain", "in": 12, "out": 3}
]
}| Field | Meaning |
|---|---|
chain_attempted | Routes in fallback order; the first is the pick. route_displays maps each to its catalog model name. |
filter_trail | How the candidate pool narrowed at each stage, from every servable route to the final chain. |
final_disposition | served, or how the request ended otherwise. |
The same record is shown in the portal under Observability › Logs, rendered with model names. See Reading a routing decision.
Models
| Method and path | GET /models |
| Purpose | The models available for inference. The first entry is auto, so routing is selectable in any client that builds its model picker from this list. With a key, the project's provider-key models are appended. |
| Authentication | None |
{
"object": "list",
"data": [
{"id": "auto", "object": "model", "owned_by": "inferbase", "name": "Inferbase Auto - Smart Routing"},
{"id": "openai/gpt-oss-120b", "object": "model", "owned_by": "inferbase"}
]
}Model health
| Method and path | GET /models/{model_id}/health |
| Purpose | Whether the provider that serves the model is reachable. |
| Authentication | None |
{"model": "openai/gpt-oss-120b", "status": "ready"}status is ready, loading, or unavailable. It is a liveness probe against the provider's API, not a cold-model detector: a warm and a cold model both report ready, and loading means the provider's API is not answering. Polling it does not warm anything.
Last updated September 10, 2026.