RetailOpinion
AI Shopping Agents Fail 43% of the Time. Most Failures Still End in a Purchase.
"AI agents have the potential to power the next era of commerce, but trust is key to making it work."

The brief
- Visa's test of 22 AI shopping models revealed a failure rate of 43%, with models passing only 57.2% of runs.
- Despite failures, 54% of the time the AI still completed a purchase, indicating a potential for commerce despite trust issues.
- The best-performing AI model passed 82.2% of runs, while the worst passed only 28.9%; models that better uncovered hidden buying factors performed significantly better.
AI agents are being built to shop on people's behalf, but a new test from Visa found they often get it wrong. On November 5, 2026, Visa released the Visa Agentic Commerce Benchmark, which put 22 AI models from 12 labs through 15 realistic shopping challenges.
The brief
- Visa's test of 22 AI shopping models revealed a failure rate of 43%, with models passing only 57.2% of runs.
- Despite failures, 54% of the time the AI still completed a purchase, indicating a potential for commerce despite trust issues.
- The best-performing AI model passed 82.2% of runs, while the worst passed only 28.9%; models that better uncovered hidden buying factors performed significantly better.
The models passed 57.2% of runs. In about 54% of the runs they failed, the AI still made a purchase.
"AI agents have the potential to power the next era of commerce, but trust is key to making it work," said Rubail Birwadker, global head of growth at Visa.
How the Test Works
The benchmark covers categories including TVs, laptops and personal care appliances. To pass a run, a model has to identify a shopper's needs through conversation, uncover every requirement in the task, and make the right decision to buy or not buy. If no product fits, the AI is expected to decline.
In one scenario, a model shopping for a TV had to find out the customer's budget, format compatibility, whether the set supported HDR10+ and its gaming capabilities.
Visa said the requirement to uncover enough information before deciding guards against lucky guesses.
Failures included picking a product that did not meet the customer's needs and buying before finding all of the customer's requirements.
Each model had three attempts at every scenario. All three attempts passed only 34.2% of the time, and models often failed when repeating scenarios they had already passed, according to The Next Web.
Which Models Did Best
Results varied widely. The best-performing model passed 82.2% of runs, and the worst passed 28.9%.
Visa did not name the models in its announcement. According to Fortune's reporting, cited by The Next Web, agents from Google and Meta shared the top score of 82%, a Claude agent came third with 76%, and an OpenAI agent placed sixth with 64%.
Models that were better at uncovering hidden buying factors performed better overall. The top five on that measure passed 70.2% of runs, compared with 44.4% for the bottom five.
Cost also mattered: the five most expensive models per run passed 67.1% of runs, against 50.2% for the five least expensive.
"We're not there yet from a trust or technology perspective," Birwadker told Fortune. "But we'll soon get into a world where consumers can trust and delegate."
The benchmark is publicly available on Visa Payments Frontier, where users can take the test and watch replays of AI attempts.
Visa said it has shared the benchmark with leading model providers, and plans to add more scenarios and product categories and extend testing to checkout, payment and order placement.
United Restricts Agents
Days after Visa's results, United Airlines announced on November 11 that it is restricting access for third-party AI agents, both on its own digital platforms and through online travel agencies and aggregators.
United said the aim is "to prohibit use by unauthorized, third-party personal AI agents or assistants," according to Skift. It is the first US airline to explicitly restrict them.
"I don't want to create another middleman," CEO Scott Kirby said at a Chicago event, as reported by The Wall Street Journal.
United said sharing account credentials with third-party agents can expose sensitive information, and that AI tools could trigger unintended purchases, account changes or MileagePlus redemptions.
It also said automated purchases could conflict with FAA and Department of Transportation rules, pointing to check-in, where a person must confirm their bags contain no prohibited items.
United said it will evaluate the policy as personal AI assistants evolve.
Trust Before Checkout
Not every airline is taking United's approach. Delta CEO Ed Bastian has said he is "cautious" about AI agents in booking, while Southwest has launched a ChatGPT plug-in for flight search.
Consumers are also hesitant. Visa's Trust Index, released in September, found 72% of US consumers had used an AI assistant, but only 23% trusted AI to make payments for them.
Visa's next round of testing will move closer to the point of payment. The company plans to add more shopping scenarios and product categories, and to extend the benchmark to checkout, payment and order placement.
Birwadker said that stage is coming. "We'll soon get into a world where consumers can trust and delegate," he said.
Join the discussion
0 answers
People who run these systems answer here. Your verified role is shown, never your employer unless you choose.
Sign in with LinkedIn to answer
Read next
Chaos Predictions — every Monday
Where enterprise AI is heading next, read from the week’s deployments across banking, pharma, manufacturing and retail.
Free. One email every Monday. Unsubscribe in one click.












