New AI test finds US leaders ahead of Chinese rivals by more

A new way of scoring AI models makes the leading American systems look further ahead of Chinese rivals than before. Artificial Analysis, an independent firm that measures AI performance, rebuilt its testing system. Under the old version, the best open-weight models trailed the overall leader by 3 points. Under the new one, they trail by 12.

What changed

Open-weight models are AI systems whose inner workings are released publicly, so other businesses can download and adapt them. Many come from Chinese companies, including Moonshot's Kimi K3.1 Pro. Artificial Analysis says many of its tests are now kept secret. That makes it harder for AI makers to train a model to do well on a known test. The firm also changed the tasks and how it weights them.

On the new scale, the best open-weight models scored 32. Anthropic's Claude Opus 5.5, then the overall leader, scored 58. Even Claude Haiku 5.5, a smaller Anthropic model, beat every model with publicly released weights.

Was the old test too generous?

Sridhar Ramaswamy, CEO of Snowflake, put it bluntly on X: the gap jumped the day the tests went private. He asked whether open models were really catching up, or whether the test was leaking. Microsoft CEO Satya Nadella and other experts call this problem "benchmark hacking." Even Nathan Lambert of Trillium Labs, who researches open-weight models, said the old index was flattering them too strongly.

What the usage numbers say

Another check is how much people use these models. Dax Raad of OpenCode looked at traffic on OpenRouter, a service that lets developers reach models from many providers. He estimated that almost all of the previous day's DeepSeek traffic there could run on one rack of powerful Nvidia computers, costing roughly $6 million to $7 million.

That is a small amount given the attention Chinese models get. But OpenRouter is only a slice of the market. ChatGPT alone has more than 1 billion monthly users, and much of that activity would not show up there.

Why it matters

Chinese models were seen as a threat because they are often cheaper, which could force OpenAI and Anthropic to charge less. The new scores suggest that threat may have been overstated. That could help the case for Anthropic's planned IPO and a possible future OpenAI listing. Lambert still says open-weight models are very competitive in software development. They can also give businesses more control and potentially lower costs.

What to do

• Treat any AI ranking as one input, not a verdict.
• Test a few models on your own real tasks.
• Compare costs as well as scores before you commit.

It is not yet clear how much of the wider gap comes from secret tests and how much from the changed tasks and weights.