Cloudflare's new AI picker chooses a model for you, and it can misjudge risk

Cloudflare has released a tool in public beta that chooses which AI model answers each request, so you do not have to. It is called Auto Router and runs through Cloudflare's AI Gateway. Cloudflare's own test shows it scoring about ten points below the top model on success rate, at roughly a third of the cost. One practitioner warns it could misjudge how risky a task is.

What it does

You set the model to cloudflare/auto. A classifier, which is a program that sorts requests, then reads each one and picks a model. It rates the request from 1 to 5 on four things: complexity, ambiguity, stakes, and how much it depends on earlier messages. It also sorts the request into one of 14 task types. It looks only at the newest turns of a conversation, capped at 8 KiB of text.

The trade-off in the numbers

Cloudflare tested 97 tasks, with three samples per model. Against Claude Opus 5.5, the router came in ten points lower on success rate while spending about a third as much. Cloudflare published a result where its router loses, which helps teams judge the trade for themselves. One caution: the two baseline models in that table are not in the default pool. They can be reached only through opt-in headers, so anyone repeating the test should check what their gateway actually picks.

Where the risk is

Igor Pyrin, a tech lead who has worked on banking and healthcare platforms, raised the sharpest concern. In a long session, a user might say early on that something is a customer refund, then later just write "go ahead." Because the router leans on recent turns, that short message could look low stakes. Pyrin suggests an application-owned risk label that carries across turns and decides whether a request can use the auto option at all.

Controls and limits

• Responses show which model was chosen and why, so choices can be reviewed later.
• You can limit the allowed models or providers per request, but not as saved profiles.
• Without a session ID, one conversation may hop between models and lose prompt caching.
• WebSockets are not supported. Chat Completions and the Responses API are.

Critics on Reddit argue that auto routers can cost more for less intelligence. Cloudflare also says its routing weights come from benchmark results, so new models must be added to the scoring by hand.

What happens next

Auto Router is free during the beta. No price has been stated for afterward. Cloudflare also plans a version called cloudflare/auto-best, which uses the same classifier and pool but picks the highest expected quality with no cost tradeoff. Cloudflare says the model pool may change over time.