Tools and MCP

Tool calling

tools and tool_choice take the OpenAI shape and are forwarded to the provider. Tool calls come back in message.tool_calls and, when streaming, as delta.tool_calls fragments with finish_reason: "tool_calls".

What Inferbase adds:

• With model: "auto", routing is limited to models whose tool support is confirmed and whose tool-use quality is measured, and the served model is disclosed. A model whose tool support is unknown is excluded, because a silently mangled tool call is worse than a smaller pool.
• With a pinned model or a routing.model_pool, the pool is walked in your stated order over its tool-capable models.
• A turn whose history shows tool activity counts as a tool request even without a tools array, and a conversation mid tool loop stays on the model that made the call.
• tool_choice of required or a named function is read and disclosed. A request that forces a call is served only by deployments that honor the field: each deployment is asked, with a required call, and one that answers in prose is not eligible for such requests (the decision's tool_choice_fit step). A served answer that still comes back without a call fails over to the next deployment of the same model when one is in the chain, and the decision says tool_choice_honored: false when the final answer carried no call. On an Anthropic-served model with thinking on, it is served as auto and disclosed as tool_choice_downgraded, because the Messages API rejects the combination.
• A routing policy can allow-list the tool names a key may send, cap what one conversation may spend in steps or dollars, and place a prompt-cache breakpoint on the tool block. See Tool governance.

MCP

MCP clients flatten a server's tools into ordinary function tools before calling a model, so an application using MCP servers routes through Inferbase like any other tool-using application. Three levels are available.

1. Nothing to configure

Keep the MCP connection in your application. Inferbase routes the request and governs the tools: a tool-carrying request reaches only models that support tools, the policy's allow-list applies, task rules can prefer models for particular tools, the conversation stays on one model for the life of the loop, and the step and spend caps bound a loop that does not terminate. The tool names your projects send are recorded, so the policy editor offers them.

2. Declare the servers

Because a client merges every server's tools into one array, create_issue and delete_branch reach Inferbase side by side with no record of which server each came from. Declare the servers under Gateway › Connections › MCP servers and a policy can allow tools by server, and a tool from no declared server shows up as drift on the agent's tool surface.

3. Inferbase calls the tools

Give a declared server its endpoint and a credential and Inferbase becomes the MCP client: it fetches the server's tool list, injects it, executes the calls the model asks for, and loops until the model has an answer. Four things become checked rather than reported: the tool list is what the server exposes, the allow-list is enforced against each call, tool latency is measured, and stopping a conversation stops it mid tool call.

Limits of a brokered run:

• Only remote MCP servers over HTTPS are reachable. A server running on a developer's machine is not.
• The credential is a static token, sent as Authorization: Bearer <token> on every call to the server. Servers that require an OAuth sign-in flow are not supported yet; use a server that accepts a personal access token or an API key. Saving or replacing a token fetches the tool list again, so a wrong token shows on the row at once.
• Tool results are passed to the model as text. A result that returns an image or an embedded resource reaches the model as a placeholder such as [image content], so the model can say what it received but cannot see it.
• A run is several model calls with tool execution between them, so it cannot be streamed. A streaming request is refused rather than served without its tools.
• Each turn is routed, priced, and recorded separately. The response carries the whole run's tokens, cost, and wall time, plus a usage.broker block: how many turns, which tools were called (tools_executed), which of those calls failed (tool_errors and tools_failed), which servers offered tools (servers), and why the run stopped. A completed answer and a run cut off at its turn ceiling are the same shape of response.
• A tool call that fails does not by itself end the run. The model is told why and can try another tool, so check tool_errors when an answer looks thinner than expected. When a server refuses its credential during a run, the server's row under Gateway › Connections › MCP servers shows the refusal until the next successful refresh.
• Runs are bounded by the session step and spend caps, a turn ceiling, a deadline, and a limit on tool failures, whichever comes first. A server that refuses its credential stops the run at once, since no retry can fix that; any other failure of one tool may happen three times in a row (fewer if the policy says so), and the third ends the run. usage.broker.stopped says which bound ended a run: complete, max_turns, deadline, tool_errors, credential_refused, foreign_tool or disconnected. If the caller disconnects, the run is abandoned rather than left executing tools.

Last updated September 10, 2026.