Chat completions · Docs


Method and path	`POST /chat/completions`
Purpose	Create a chat completion, streaming or not. `model: "auto"` lets Inferbase choose the model; a model id pins it.
Authentication	API key or session
Limits	Request body up to 1 MiB, base64 images and audio included. `max_tokens` up to 128,000. Up to 200 messages and 128 tools.

Request

Plain messages serialize exactly as they do for OpenAI. Every field the OpenAI chat completions API documents is accepted: forwarded when every provider accepts it, mapped when Inferbase has an equivalent, or accepted and reported as not forwarded.

Field	Type	Meaning
`model`	string, required	`auto`, or a model id from `GET /models`.
`messages`	array, required	Message objects with `role` and `content`. Content is a string or an array of blocks: `text`, `image_url` (with `detail`), `input_audio` (base64 with `wav` or `mp3` format), or `video_url`. Assistant messages may carry `tool_calls`; `tool` messages need `tool_call_id`.
`stream`	boolean	Server-sent events. Default false.
`temperature`, `top_p`, `stop`	as OpenAI	Forwarded.
`max_tokens`	integer	Defaults to 4,096 when omitted, so long output is truncated with `finish_reason: "length"` unless raised.
`max_completion_tokens`	integer	The newer name; used when `max_tokens` is absent.
`n`	integer	Only 1. A routed request picks one model for one answer; `n > 1` is a 422.
`seed`, `presence_penalty`, `frequency_penalty`, `logit_bias`, `logprobs`, `top_logprobs`, `user`, `parallel_tool_calls`	as OpenAI	Forwarded. Anthropic-served models have no equivalent for `seed`, the penalties, `logit_bias`, `logprobs`, or a message `name`: a request pinned to one is refused with a 400 naming the parameter, and a routed request that lands on one lists what was not forwarded in `usage.dropped_params`.
`reasoning_effort`	string	`low` and above turn extended thinking on where the model has it. `none` and `minimal` do nothing.
`response_format`	object	`text`, `json_object`, or `json_schema`. See Structured output.
`tools`, `tool_choice`	as OpenAI	See Tools and MCP.
`stream_options`	object	`{"include_usage": true}` ends the stream with a chunk carrying `usage` and an empty `choices` array.
`metadata`, `store`, `prompt_cache_key`, `safety_identifier`, `service_tier`, `verbosity`, `modalities`, `prediction`, `web_search_options`, `audio`	as OpenAI	Accepted for SDK and agent-framework compatibility, never forwarded. The ones a request set are listed in `routing.user_rules_applied.accepted_not_forwarded`. `metadata.session_id`, then `metadata.trace_id`, then `prompt_cache_key` name the session when `routing.session` is absent.
`routing`	object	The Inferbase extension. See The routing object.
`extended_thinking`	boolean	Inferbase extension: return the model's reasoning trace where the model supports it.

A field not in the OpenAI specification and not an Inferbase extension is a 422 naming it.

The routing object

All fields optional. A request may narrow what its routing policy allows and never widen it; see Routing policies.

Field	Type	Meaning
`optimize`	string	`balanced`, `quality`, `cost`, or `latency`. Overrides the policy's objective unless the organization enforced it.
`preset`	string	`strict`, `standard`, or `permissive`: the evidence bar a model must clear. Same enforcement rule.
`model_pool`	array of model ids	A subset of the policy's pool for this request. A model outside the pool is refused with a 403 naming the allowed set. Order matters: with tools, or when the policy walks the list in your order, the first is tried first.
`scope`	string	`inferbase` (managed models), `keys` (the project's provider keys), or `all`. May be narrower than the policy's scope, never wider. Ignored when `model_pool` is set.
`task`	string	Overrides the classifier's task: `coding`, `math`, `knowledge_qa`, `analysis`, `creative_writing`, `summarisation`, `translation`, or `general`. An unrecognized value is ignored. A valid override also fixes complexity at neutral.
`session`	string	Your identifier for a conversation, 1 to 128 printable characters with no spaces. Turns that share it are routed as one conversation, group under one run in Usage, and are subject to the policy's session limits. Also accepted as the `X-Inferbase-Session` header; the body wins if both are sent. Without one, a conversation is recognized by its opening system prompt and first user message.
`session_affinity`	boolean, default true	Whether a conversation stays on its model. A turn moves only when that model can no longer serve the request or when a fresh pick beats it by more than the objective's own margin. Set false to route every turn independently.
`require_prompt_cache`	boolean, default false	Serve only on a provider that caches prompt prefixes. A pool where nothing caches is refused with a 503 saying so. Without it, a prompt carrying `cache_control` breakpoints already prefers such a provider when one is within the routing band.

Timeouts

Each attempt has a first-token budget and the request has a total deadline; the routing policy sets both. The budget is measured to the first token, not to completion, so a long answer is never cut off for being long. A conversation mid tool loop falls back only within the model that made the call.

Structured output

response_format takes {"type": "json_object"} for JSON, with the prompt also asking for JSON, or {"type": "json_schema", "json_schema": {...}}, forwarded to the provider as written. Routing serves only models with verified structured-output support.

When a non-streaming answer arrives wrapped in code fences or commentary, the JSON is taken out of it and usage.structured_output_repair says so. Nothing is rewritten to make it parse. The routing policy can turn extraction off, or have the model asked once more when there is no document to take.

Example

bash

curl https://api.inferbase.ai/api/v1/inference/chat/completions \
  -H "Authorization: Bearer inf_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is machine learning?"}
    ],
    "temperature": 0.7,
    "max_tokens": 256,
    "routing": {"optimize": "quality"}
  }'

Response

The OpenAI shape. Two fields carry Inferbase meaning.

model is the model that served the request, as provider/model-id. It never says auto.
id is the request id, the handle for GET /routing-decisions/{request_id}.

Non-streaming responses also carry the served model in the X-Inferbase-Model header.

json

{
  "id": "req-9f2c1e...",
  "object": "chat.completion",
  "created": 1712345678,
  "model": "openai/gpt-oss-120b",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "Machine learning is a subset of artificial intelligence..."},
      "finish_reason": "stop"
    }
  ],
  "usage": {"prompt_tokens": 24, "completion_tokens": 128, "total_tokens": 152}
}

usage may carry dropped_params, structured_output_repair, cache token counts, and, for brokered tool runs, a broker block.

Streaming

With stream: true the response is server-sent events: OpenAI-shaped chunks wrapped by three Inferbase events in a fixed order.

A routing object, before any content, carrying the decision. See Routing.
The chunks. delta.reasoning_content carries thinking passthrough, delta.tool_calls carries tool call fragments, and the last chunk carries finish_reason.
A usage event, then [DONE].

A failure after the stream has started arrives as an event shaped {"object": "error", "error": {...}} followed by [DONE], with the routing disposition in code. Clients that do not know the Inferbase events skip them.

Errors

Code	When
400	A parameter the pinned model cannot honor, naming it.
402	Insufficient credit, or the conversation reached its policy's spend cap.
403	The key's policy refuses the request: a tool outside the allow-list, a pool or scope wider than the policy, an enforced objective or preset, a pinned model outside a locked pool.
413	The body exceeds 1 MiB, or no pool model has a large enough context window.
422	Validation, an unknown field, `n > 1`, or a reused idempotency key with a different body.
429	Rate limited, or the conversation reached its policy's step cap.
503	No eligible model for the request's tools, modality, structured output, or quality floor, with the reason.

See Errors for the full list and the envelope.

Last updated September 10, 2026.

PreviousAuthentication and idempotency NextTools and MCP