Tools and MCP

Tool calling

tools and tool_choice take the OpenAI shape and are forwarded to the provider. Tool calls come back in message.tool_calls and, when streaming, as delta.tool_calls fragments with finish_reason: "tool_calls".

What Inferbase adds:

• With model: "auto", routing is limited to models whose tool support is confirmed and whose tool-use quality is measured, and the served model is disclosed. A model whose tool support is unknown is excluded, because a silently mangled tool call is worse than a smaller pool.
• With a pinned model or a routing.model_pool, the pool is walked in your stated order over its tool-capable models.
• A turn whose history shows tool activity counts as a tool request even without a tools array, and a conversation mid tool loop stays on the model that made the call.
• tool_choice of required or a named function is read and disclosed. On an Anthropic-served model with thinking on, it is served as auto and disclosed as tool_choice_downgraded, because the Messages API rejects the combination.
• A routing policy can allow-list the tool names a key may send, cap what one conversation may spend in steps or dollars, and place a prompt-cache breakpoint on the tool block. See Tool governance.

MCP

MCP clients flatten a server's tools into ordinary function tools before calling a model, so an application using MCP servers routes through Inferbase like any other tool-using application. Three levels are available.

1. Nothing to configure

Keep the MCP connection in your application. Inferbase routes the request and governs the tools: a tool-carrying request reaches only models that support tools, the policy's allow-list applies, task rules can prefer models for particular tools, the conversation stays on one model for the life of the loop, and the step and spend caps bound a loop that does not terminate. The tool names your projects send are recorded, so the policy editor offers them.

2. Declare the servers

Because a client merges every server's tools into one array, create_issue and delete_branch reach Inferbase side by side with no record of which server each came from. Declare the servers under Gateway › Connections › MCP servers and a policy can allow tools by server, and a tool from no declared server shows up as drift on the agent's tool surface.

3. Inferbase calls the tools

Give a declared server its endpoint and a credential and Inferbase becomes the MCP client: it fetches the server's tool list, injects it, executes the calls the model asks for, and loops until the model has an answer. Four things become checked rather than reported: the tool list is what the server exposes, the allow-list is enforced against each call, tool latency is measured, and stopping a conversation stops it mid tool call.

Limits of a brokered run:

• Only remote MCP servers over HTTPS are reachable. A server running on a developer's machine is not.
• A run is several model calls with tool execution between them, so it cannot be streamed. A streaming request is refused rather than served without its tools.
• Each turn is routed, priced, and recorded separately. The response carries the whole run's tokens plus a usage.broker block: how many turns, which tools ran, and why it stopped. A completed answer and a run cut off at its turn ceiling are the same shape of response.
• Runs are bounded by the session step and spend caps, a turn ceiling, and a deadline, whichever comes first. If the caller disconnects, the run is abandoned rather than left executing tools.

Last updated September 10, 2026.