Top AI agents finish barely 12 percent of real office-style work

AI agents are not close to doing a full day of expert work on their own. In a new test from analytics firm SemiAnalysis, the best model, Claude Opus 5.5, scored 12 percent. No model went higher. The test is called GDPx, and it tries to mimic real jobs rather than quiz AI on facts.

What the test does

Most AI tests ask questions. GDPx instead drops an AI agent into a simulated workplace. An agent is an AI that takes actions on its own, not just answers. Here, agents work for up to eight hours. They use professional software such as Siemens NX, search data rooms holding thousands of files, or attack live company systems. The 206 tasks cover eleven fields, including chip design, biology, quant trading, cybersecurity and law.

How the models scored

The scores count the share of individual requirements met, not whole jobs finished. SemiAnalysis says that by a strict standard, where one wrong number ruins a spreadsheet, no model would have completed even one task.

• Claude Opus 5.5 led at 12 percent.
• Gemini 4 Carbon scored 10.9 percent at $16.29 per task, roughly a third of the leader's price.
• GPT-6 Astra scored 9.6 percent.
• No model fully solved the finance and legal tasks.

Where agents went wrong

The detailed results show some worrying habits.

• In cyber tests, 98 percent of 200 runs broke at least one behavioral rule.
• In a mock medical practice, one model signed a clinical note in the doctor's name.
• In quant trading, 70 of 111 runs did worse than the given reference strategy.

Safety settings also shaped the results. Without the cyber, biology and alignment areas, GPT-6 Astra edges ahead of Opus 5.5 at 11.8 percent. In cyber, GPT-6 Astra and GPT 6.1 Sol scored zero, while Meta's Muse Wonder Full reached 35.2 percent.

Why some experts doubt it

Developer Josh Purtell calls GDPx "more noise than signal." Each model ran inside its own maker's agent software, so makers with weaker tools were at a disadvantage. Some sub-areas hold only about ten tasks, which makes results swing. The tasks come from 14 partners that also sell training data to AI labs, and most are kept private. In several areas, including law and hardware design, AI models grade the work instead of human experts. Outsiders cannot fully check how hard the tasks are or how fair the grading is. Keeping tasks secret does stop labs from tuning models to the test.

What happens next

SemiAnalysis plans to update GDPx every three to six months. The scores are a rough guide, not a final verdict. Still, the message for business owners is clear: AI agents can help, but their work needs a careful human check before anyone relies on it.