We Tested Reasoning Controls Across 97 AI Model Routes. Here's What Broke.

We sent the same thinking-off request to 97 production model routes, every managed chat route our gateway serves across three serverless providers, and read each answer for evidence of reasoning. On 10 of them the standard control did not stop the model: seven reasoned whatever was sent, and three ignored the usual spelling while obeying a less common one. Twelve more served reasoning-capable models with thinking off by default and never reasoned unless a request explicitly asked. One route rejected the standard control outright, and four were inconclusive. Taken together, 28% of routes behaved differently from what a developer sending the conventional switch would expect, and in the 22 silent cases nothing in the response distinguished a route that honoured the switch from one that discarded it.

This is more than an API compatibility problem. Reasoning tokens are billed at the output rate, and in the 30 days to 2026-11-06, across 8,672 answers on our platform where thinking stayed on, reasoning made up 79% of the billed output. A switch that does not reach the model changes the cost of a request several times over, lengthens the wait for the first visible token by the whole reasoning phase, and, with a small output budget, can leave the caller with a successful response that contains no answer at all. This post covers why the controls exist in so many spellings, what the 97 routes did with them, what an ignored switch costs, and what we changed in our gateway as a result.

Five ways to say "do not think"

Every reasoning model exposes a mode toggle somewhere, and the location is a matter of lineage rather than standard. The OpenAI shape is a top-level reasoning_effort whose supported values are model-dependent and include none. Anthropic's Messages API carries a thinking object with its own type values, and DeepSeek's API borrowed that shape, documenting thinking: {type: "disabled"} as the off switch. The Qwen3 family introduced enable_thinking, a boolean that lives in the chat template and is on by default.

That last detail matters more than it looks. A chat template is the piece of code that turns a list of messages into the token sequence the model reads, and on open-weights models it is where the thinking toggle is implemented: the template emits or omits the tokens that open a reasoning block. Serving stacks therefore expose the flag through whatever mechanism feeds extra variables into the template. In vLLM that is chat_template_kwargs, and the vLLM documentation is candid that the key inside it differs by model family: Qwen3 reads enable_thinking, Gemma 4 reads the same key with the opposite default, and some other families read thinking instead.

A provider that fronts several model families with one OpenAI-compatible endpoint has to translate between these shapes, and each provider has made its own choices about which top-level fields map onto which template variables. The result is five spellings in common use for the same intent:

None of them is wrong, since each is the native shape somewhere. The question for anyone sending requests through an OpenAI-compatible endpoint is which of them the route in front of them actually reads.

How we measured

The measurement was deliberately plain, and its limits are worth stating up front because the conclusions depend on them. On 2026-10-16, for each of the 97 active managed chat routes we serve across three providers, we built the request exactly as our gateway builds it and asked a short reasoning-prone question (the bat-and-ball problem) with max_tokens set to 400 and the provider's default sampling settings. We asked it twelve ways: with no thinking field at all, with the off switch our gateway sent at the time, with each of the five off spellings sent alone, and with each of the five matching on spellings sent alone. Each cell was asked once, sequentially per route, three routes at a time per provider.

Every answer was read for reasoning, and the signals are not equally strong. We treat three as confirmed reasoning: a reasoning_content or reasoning field on the message, an inline block delimited by think tags in the content, or a reasoning_tokens count in the usage object. A fourth signal, the whole output budget consumed with no visible content, counts as suspected reasoning. We included it because at least one provider bills reasoning tokens and returns none of the text, so a check that ignored the signal would call that route a non-reasoner, but a serving or formatting fault can produce the same symptom, and a reader weighing the results should know that the cap-exhausted cells rest on inference rather than evidence.

Two further limits. A single request per cell cannot distinguish a route that always behaves one way from a route that behaves that way most of the time, and the per-route checks we built afterwards confirm any change with a second request for exactly that reason. And the 400-token cap was chosen so that reasoning would show before it was cut; a budget too small for the reasoning to appear would make a brief reasoner look like a non-reasoner.

Throughout this post the providers are called A, B and C. All three serve our production traffic, and the point of the measurement is how the serverless category behaves rather than any one partner's choices. Every statement about how an endpoint handled a field describes the routes we tested on that day, not a property of the provider's platform or of the serving software in general.

What 97 routes did with the same request

The outcomes sorted into seven groups:

The first two rows are the comfortable cases, and together they cover 72% of routes. The other 27 separate into four findings that mean different things.

Ten routes could not be reliably switched off with the standard control. Seven reasoned under every spelling we sent, and three ignored the standard spelling while a different one worked. MiniMax M3 on Providers B and C ignored both reasoning_effort: "none" and enable_thinking: false but stopped on thinking: {type: "disabled"}, the DeepSeek-style shape. In all ten cases the response was a normal 200 with a well-formed answer.

Twelve routes never reasoned unless told to. These serve reasoning-capable models (the DeepSeek V3.1 and V4 family and Gemma 4 on Provider A; Gemma 4 and Kimi K2.5 on Provider B) with thinking off by default, and an on spelling such as reasoning_effort: "high" switched them on. A client that only ever sends off switches, which until October described our own gateway, never sees these routes reason at all. The consequence is subtle: any quality estimate that assumes the model's published thinking score overstates what such a route delivers, and a router that ranks on that estimate will prefer it for the wrong reason. These twelve also answered with a 200 every time, since there was nothing to refuse.

One route rejected the standard control. Provider C's Qwen3.5-9B refused reasoning_effort with an input validation error, so every thinking-off request to it failed rather than silently succeeding, which is at least honest. And four routes were inconclusive: they rate-limited the baseline request, so we could not establish their default and did not count them either way.

Across the whole table one pattern held. The working switch is a property of the route, meaning the pairing of a provider and a model, rather than of the provider alone. Provider A honoured reasoning_effort and the vLLM-style chat_template_kwargs.enable_thinking on every reasoning model it serves, and silently ignored the bare enable_thinking, the thinking template key and the thinking.type object. Provider B ignored reasoning_effort: "none" on its Qwen and GLM 4.x routes while obeying it on DeepSeek, Kimi K3 and GLM 5.x. The same Kimi K2.6 obeyed the off switch on Provider A and ignored it on Provider B. A rule written per provider would have been wrong on roughly a third of that provider's routes.

One model family, three providers, three answers

The clearest way to see the pattern is to hold the model fixed and vary the provider. Qwen3.5 is served by all three, and the five off spellings produced a different answer sheet on each.

Only one spelling, the vLLM-native chat_template_kwargs: {enable_thinking: false}, stopped reasoning on all three. That is consistent with how the layers are arranged: it bypasses the provider's translation and speaks to the chat template directly, so it works wherever the provider forwards template arguments and the template reads that key. The OpenAI-shaped reasoning_effort, which is what most SDKs, gateways and tutorials reach for first, worked on Provider A, was ignored on Provider B and was rejected on Provider C. A developer testing against one provider and shipping against another would have had no warning that the behaviour changed.

The on direction showed the mirror image, with one more wrinkle. Provider B returned reasoning text under reasoning_effort: "high" from two models that are not reasoning models at all, its DeepSeek V3 and Llama 4 Maverick routes. The switch had been wired to a template path that makes any model write a reasoning block whether or not it was trained to use one. We recorded it and do not rely on it, but it is a reminder that an accepted parameter and a meaningful parameter are different things.

Where the switch is actually read

The mechanism behind all of this is simple once stated. A request to an OpenAI-compatible endpoint passes through three layers before a model sees it, and the thinking toggle can be interpreted, forwarded or dropped at each one.

The first layer is the provider's API front end, which receives the JSON body. On the routes we tested, an unrecognised field was accepted rather than rejected, which is a common choice for compatible endpoints because strict validation breaks the long tail of client libraries that send extra keys. This is where reasoning_effort and enable_thinking either get translated into something the serving stack understands or get discarded.

The second layer is the serving stack, which applies the chat template. Template arguments arrive through a mechanism like vLLM's chat_template_kwargs, and the template reads whichever variable names its author chose. As the Hugging Face chat template documentation shows, two models fine-tuned from the same base can use entirely different control tokens, and the same is true of the variables that gate a reasoning block.

The third layer is the model, which reads the rendered prompt. If the template opened a reasoning block, the model reasons. If it did not, the model may still reason on its own initiative if that is how it was trained, which is one plausible reading of the seven routes that could not be stopped: the template had no off path, or the provider never exposed it.

Given that chain, the measured results stop being surprising. A provider's translation layer maps a finite set of top-level fields, and if the mapping is written once per provider rather than once per model family, it fits some templates and not others. The direct template route works more often because it skips the translation. And nothing in the chain reports back which layer handled the field, so the client learns nothing from a successful status.

What an ignored switch costs

The cost of an ignored off switch is the reasoning itself, billed at the output rate. How large that is depends on the question and the model, and the honest way to put a number on it is to measure it on real traffic rather than on a benchmark prompt.

Across our platform in the 30 days to 2026-11-06, 8,672 successful answers ran with thinking on. Reasoning accounted for 79% of all billed output tokens across those answers. On Qwen3.8 Max, the most used model among them, the average answer carried 700 output tokens of which 615 were reasoning, so the visible reply was 12% of what was paid for. Qwen3.8 Flash averaged 1,672 output tokens, 1,456 of them reasoning. The pattern held across families: of the 36 models with at least 50 thinking-on answers in the window, 23 spent more than 80% of their output tokens on reasoning. A few at the other end barely reasoned at all despite the mode being on, which is the default-off problem from the previous section seen from the billing side.

A worked example makes the multiplier concrete. Take a classification task that needs a 20-token answer, on a model priced at $2.00 per million output tokens. With thinking off, 10,000 requests cost $0.40 in output. If the off switch is ignored and the model reasons for 600 tokens before each 20-token answer, which is in line with the averages above, the same 10,000 requests cost $12.40 in output, a 31x difference on the output side of the bill. The input side is unchanged, so the total bill does not move by 31x, but on short-answer workloads output is where the cost lives. The qualification is that reasoning is not free of value: on a simple classification the quality difference between modes is usually negligible, while on mathematics or multi-step coding it can be substantial, so the saving is real only where the task did not need the reasoning in the first place. This is the hidden factor that falling per-token prices do not show: the price per token fell, and the number of tokens per answer quietly multiplied.

The second cost is not money but silence. A thinking model given a small max_tokens can spend the entire budget reasoning and stop before the visible answer starts. In a load test on 2026-10-12 with max_tokens set to 400, 119 of 464 non-streamed routed requests, 26%, had landed on a thinking model that spent the whole budget reasoning, and came back with HTTP 200, finish_reason: "length" and an empty content. In a routing benchmark four days later with the budget raised to 2,048, 66 of 480 answers still stopped at the cap, and on one factual-recall arm 22 of 70 answers from a single model were reasoning runs cut off with nothing visible. The request succeeded by every metric a dashboard shows, and the user received nothing.

The third cost is time. The first visible token of a thinking answer arrives after the reasoning is finished, so time to first token on a route that ignores the off switch is the reasoning length multiplied by the model's output speed. For a non-streamed request that wait is the whole response. A latency budget set for a non-thinking model is wrong by the entire reasoning phase on a route that reasons anyway.

Why a 200 tells you nothing

Everything above points at one operational conclusion: the only way to know what a route does with a thinking switch is to send a request and read the answer. Status codes, documentation and the provider's model page all describe intent; the answer describes behaviour.

Reading the answer means checking the signals described in the method section, and weighting them accordingly: a reasoning field, a think block or a stated reasoning-token count is proof, and an exhausted budget with nothing visible is a strong hint that deserves a second request with a larger cap. The same test run in the other direction, with an on spelling, is what catches the twelve routes that default to off. A check that only asks "does it stop" will never find a route that never started.

One more rule follows from the refusal case. A 400 on a thinking field is good information: it says the route read the field and will not accept it, so the client can retry without it and record the fact. A 200 says only that the request parsed.

What we changed: route capability profiling

The measurement reshaped how our gateway handles thinking, and the design is worth describing because any team running more than one provider faces the same question. We have come to call it route capability profiling: a route, meaning one provider serving one model, is treated as the unit whose behaviour is measured, rather than the model or the provider on its own.

The thinking switch is now learned per route rather than declared per provider. Each route's model check sends the quiet request with nothing set, reads whether the answer reasoned, and then walks the known spellings in order until one changes the outcome, confirming the result with a second request because some routes answer differently from sample to sample. The outcome is a small profile on the route: what it does by default, which spelling turns reasoning on, which turns it off, and null where nothing does. A route that needs an on switch gets one when a request wants reasoning.

The profile is refreshed on a schedule and checked against live traffic. Every answer records which mode it asked for beside the reasoning tokens it actually produced, and a route whose answers disagree with its profile for enough of a week is re-measured at once. One quiet answer flipped a profile the wrong way during the first week, so a change toward "this switch now works" needs two consistent sightings, while a change toward "this switch stopped working" is believed immediately.

The mode is disclosed and it feeds the routing decision. The routing object on every answer states whether thinking was on or off and why, including when the route ignored the request, and an answer that stopped at the cap while reasoning says so rather than returning an empty success. Ranking reads the profile too: a route that will run with thinking off is scored on what the model achieves without reasoning, which keeps the twelve default-off routes from being ranked on a score they never deliver.

The limit is worth being explicit about. Profiling a route tells the gateway what the route will do; it cannot make the route do something else. The seven routes that reason under every spelling still reason, and the profile's job there is to say so, count the reasoning against the time and token budgets, and let the router prefer a route that can honour the request when one exists. Detecting an incompatible switch is not the same as fixing it.

The broader lesson is the one we keep relearning across the catalog. A model's documentation and benchmark scores describe what the model should be capable of; the route, under a given request, determines what it actually delivers. A routing decision that reads only the model identity, the published scores and the advertised price has no way to see the difference, and this measurement is one of the clearer demonstrations that the difference is large enough to matter.

What to check in your own stack

If you send thinking switches through an OpenAI-compatible endpoint, four checks cover most of the risk.

First, for each provider and model pair you depend on, send one short request with your off switch and read the response for reasoning content and reasoning tokens. Do not infer from the status code. Repeat with your on switch if you rely on reasoning anywhere, and repeat the whole exercise when a provider updates its serving stack, because what we measured in October describes those routes on that day.

Second, look at the usage object on real traffic and compute the share of output tokens that are reasoning. If it is high on a workload that set thinking off, the switch is not reaching the template.

Third, set max_tokens with the reasoning budget included. If a route might reason, the cap has to leave room for the reasoning and the answer, or the answer will be the part that gets cut.

Fourth, prefer the spelling that reaches the template directly where the provider forwards it, and keep the provider-shaped spelling as a fallback rather than the default. The measurement above found exactly one spelling that worked across all three providers for the same model family, and it was the one closest to the model.

The suffix on a model name tells you what it can do; we covered that in instruct vs thinking vs base. What it will do on a given route, under a given request, is a separate fact, and the only way to hold it is to measure it and keep measuring. The model catalog lists the thinking capability of each model; what each managed route does with the switch is what our checks now watch for.