Visa tested 22 AI models on shopping tasks and the average pass rate was 57.2%, so letting an AI buy for you or your customers still needs a human check.
Visa has released a free online test for AI shopping helpers, and the first results show many still get orders wrong. Across 22 AI models, the overall pass rate was 57.2%. Worse, in 54% of the runs that failed, the AI still went ahead and decided to buy.
The tool is called the Agentic Commerce Benchmark. Agentic commerce simply means an AI agent shops for you, such as finding a product on a website and buying it. Visa says the test puts AI models through realistic buying challenges, including whether an agent can shop on a website. Anyone can use it online.
Visa tested 22 models from 12 providers, including Gemini, Claude, Muse and Grok. Each took on 15 shopping scenarios. The best pass rate was 82.2% of attempts. The worst was 28.9%.
Visa explains the failures this way: "Either the AI recommended a purchase that did not meet the customer's requirements, or it failed to uncover all the customer's requirements." In plain terms, the AI bought the wrong thing or did not learn what the buyer wanted.
Visa's global head of growth, Rubail Birwadker, said AI agents could power the next era of commerce, but "trust is key to making it work." For shoppers, that means checking what an AI picked before money moves. For business owners, it means an AI sending customers to your store may not match what they asked for.
Visa also predicts U.S. holiday sales will be 4.5% higher than the 2025 season. It did not give a total dollar figure. Average gift spending is forecast at $777, up 5.6% from $736. Visa says that is the strongest holiday sales growth since 2021.
Physical gifts may beat travel. Only 15% of people are prioritizing experience spending, against 27% prioritizing physical gifts. Visa found that only households earning more than $200,000 a year expected to raise both holiday airfare bookings and spending in September. Lower-income groups pulled back sharply. Visa says that could help retailers, since its holiday measure excludes travel and dining.
If you try an AI shopping helper, review the item, size, price and any other requirement before it buys. Visa's data does not say which models fall at which end of the range, so a good result on one task proves little.
Before an AI shops for you, check its pick against what you asked for, because Visa found it often buys anyway when it gets things wrong.
Mastercard describes AI tools that could buy things for you, and says trust is the hard part: who gets to spend your money, and under what rules.
Read more: Mastercard outlines how AI assistants could shop and pay for youWorld models, AI that learns physics from video, are being built into home robots and driverless cars, and your own videos may help train them.
Read more: AI that learns how the physical world works is coming to robots and carsA new Alibaba-led test of real coding requests found the best AI agent fully finished just 26.4 percent of tasks, so human review still matters.
Read more: AI coding helpers fully finish only about 1 in 4 real tasks
