How Accurate Are AI Answers About Brands? What Benchmark Edition 1 Found

How accurate are AI answers about brands? In Edition 1 of the Prime AI Visibility Benchmark, 10% of the checkable claims that AI models made about 25 well-known brands contradicted the brand's own pages (183 of 1,821). Error rates were lower with search switched on for 3 of 3 models tested both ways, and most errors were claims about free plans.

This article explains those findings for people who buy, market or sell software. Every figure comes from the published dataset and method, released on October 23, 2026. The figures describe API answers to a fixed question set collected in one week, so read them as a measured sample, not as a verdict on any AI product.

How accurate are AI answers about brands? The short answer

1. Most checkable claims matched the brand's own pages, and 10% did not. Across all seven model configurations, 183 of 1,821 checkable claims contradicted a fact sheet built from the brands' own pages.
2. Search on meant fewer errors, not zero. For 3 of 3 models run both ways, the factual error rate was lower with search on. With search on, error rates still ranged from 3.3% to 11.9% depending on the configuration.
3. Free-plan claims were most of the errors. 121 of 183 contradicted claims (66.1%) were about whether a brand has a free plan or what it includes, and they appeared in 7 of 7 configurations.
4. Cited links varied widely. Depending on the search-on configuration, 37.1%–91.4% of checked cited links supported the sentence they were attached to.

What Edition 1 measured

The benchmark sent 50 buyer questions about 25 brands in five categories (CRM software, payroll software, project management software, online travel booking and ecommerce platforms) to seven AI model configurations through their APIs, three times each. 1,050 answers came back and 1,050 were usable. Each answer was checked in four ways: which brands it named, whether its cited links supported the sentence they were attached to, whether its claims matched the brands' own pages, and whether repeated runs agreed.

For the factual checks, a dated fact sheet records 55 facts about the 25 brands across ten kinds of fact, such as free plans, plan names, headquarters, founding year and parent company. Each fact was taken from a page the brand publishes about itself, in the same week the answers were collected. A claim counts as an error only when it contradicts that page. Statements on topics outside the fact sheet were not graded, and plans or offices that the source page does not list were treated as not checkable rather than as errors. The error rate is about checkable facts, not about everything an answer said.

The configurations pair three models, each run with search on and with search off: OpenAI GPT-6 Sol, Anthropic Claude Sonnet 5.5 and Google Gemini 3.8 Flash, the last with Google Search grounding switched on and off. Perplexity Sonar (via OpenRouter) always searches, so it has no search-off pair. These are answers from each provider's API, not consumer-app sessions, and they are not live Google AI Overviews or Google AI Mode. Google appears here only as Gemini, with Google Search grounding switched on or off.

Questions were sent as written, in US English, with each provider's default settings and no system prompt. The same question asked in an app, in another market or with a follow-up can produce a different answer.

The results by configuration

The table shows each configuration's factual error rate (contradicted claims out of checkable claims) and its citation accuracy (supporting links out of checked links). A lower error rate and a higher citation accuracy are better.

Configuration	Factual error rate (claims)	Citation accuracy (links)
OpenAI GPT-6 Sol — web search on	3.3% (9 of 273)	43.5% (117 of 269)
OpenAI GPT-6 Sol — web search off	5.9% (10 of 169)	No cited links
Anthropic Claude Sonnet 5.5 — web search on	8.4% (23 of 275)	91.4% (1,152 of 1,260)
Anthropic Claude Sonnet 5.5 — web search off	11.6% (33 of 285)	No cited links
Google Gemini 3.8 Flash — Google Search grounding on	11.9% (38 of 318)	57.5% (506 of 880)
Google Gemini 3.8 Flash — grounding off	18.1% (55 of 304)	0% (0 of 1)
Perplexity Sonar (via OpenRouter) — search always on	7.6% (15 of 197)	37.1% (1,045 of 2,820)

These figures describe this question set, these brands and one collection week. They are not a ranking of AI tools, and a configuration with fewer errors here is not a more reliable source of facts about every brand. The panel is made up of well-known brands, and the results do not describe smaller or lesser-known ones.

Does search on make AI answers more accurate?

In this sample, the error rate was lower with search on for 3 of 3 models run both ways, but search did not remove errors.

Model	Search on	Search off
OpenAI GPT-6 Sol	3.3% (9 of 273)	5.9% (10 of 169)
Anthropic Claude Sonnet 5.5	8.4% (23 of 275)	11.6% (33 of 285)
Google Gemini 3.8 Flash	11.9% (38 of 318)	18.1% (55 of 304)

GPT-6 Sol had a factual error rate of 5.9% with web search off and 3.3% with it on. Claude Sonnet 5.5 had 11.6% off and 8.4% on. Gemini 3.8 Flash had 18.1% with grounding off and 11.9% with Google Search grounding on.

The search-on figures need one note each. With Google Search grounding on, Gemini decides for itself whether to search, and it searched in 64 of 150 returned answers. The answers that ran no search still count in that configuration. GPT-6 Sol searched in 150 of 150 returned answers and Claude Sonnet 5.5 in 130 of 150.

Search changed which errors remained, too. Of GPT-6 Sol's contradicted claims with search on, 9 of 9 were about free plans. With search off, the same model's contradicted claims were spread across free plans (4 of 10), headquarters (3 of 10) and plan names (3 of 10).

For a brand, the practical reading is that an answer with search on can still state a wrong fact about you. Checking whether answers cite your site is not enough; read what they say.

Free-plan claims were most of the errors

Every contradicted claim in Edition 1 fell in three kinds of fact. 121 of 183 were about free plans, 33 of 183 about headquarters and 29 of 183 about plan names. Free-plan errors appeared in 7 of 7 configurations, with search on and with search off.

They were concentrated in a few brands. Claims about the free plans of Freshsales, Asana and Ecwid together make up 111 of 183 contradicted claims (60.7%).

• Freshsales: 54 of 183 contradicted claims, in 7 of 7 configurations. Answers described a free plan, for example "Freshsales has a free plan for up to 3 users". The fact sheet, taken from the Freshsales pricing page, records: No — 21-day free trial, then a paid plan.
• Asana: 32 of 183, in 5 of 7 configurations. Asana does list a free plan, so these errors were about its terms, for example "Free for up to about 10 users". The fact sheet, taken from Asana's pricing page, records: Yes — Personal plan, $0, for one or two people.
• Ecwid: 25 of 183, in 5 of 7 configurations. Answers said a free plan exists, for example "Yes, Ecwid has a free plan". The fact sheet, taken from Ecwid's pricing page, records: No — plans are Starter, Venture, Business and Unlimited, all paid.
• Salesforce: 7 of 183, in 3 of 7 configurations. Here the error ran the other way. Answers said there is no free plan, for example "No, Salesforce does not offer a permanently free CRM plan for standard businesses". The fact sheet, taken from Salesforce's free CRM page, records: Yes — Free Suite, $0, comes with 2 user licenses.

The benchmark does not show where these descriptions came from. It shows that, in the collection week, they contradicted the page each brand publishes about its own pricing. For a buyer, that makes any AI answer about free plans, trials or user limits worth checking on the vendor's pricing page before it shapes a shortlist. For a vendor, it makes the free-plan and trial wording on the pricing page one of the first things to read in AI answers about you.

Headquarters and plan names

The other contradicted claims were about where a company is based and what its plans are called.

• Shopify headquarters: 18 of 183, in 6 of 7 configurations. Answers said, for example, "Shopify’s corporate headquarters is in Ottawa, Ontario, Canada". The fact sheet, taken from Shopify's investor FAQ, records: Las Vegas, Nevada (principal executive office).
• BigCommerce plan names: 19 of 183, in 5 of 7 configurations. Answers listed plans such as "BigCommerce offers four plans: Standard, Plus, Pro, and Enterprise". The fact sheet, taken from BigCommerce's pricing page, lists: Core, Growth, Scale, Performance.

The headquarters result is a strict reading. The benchmark checks each fact against the page named as its source, so an answer that repeats another common description of a company counts as contradicting that page. If your own pages describe your company in more than one way, AI answers may repeat any of them, and a reader cannot tell which one you consider current.

How often did cited links support the answer?

Citation accuracy, as the benchmark defines it, is the share of checked cited links that resolve to a live page and support the sentence they are attached to. Dead links, links that redirect to a homepage and links whose page does not support the sentence count against it. Links that could not be checked are reported separately and left out. A link attached to several sentences counts once for each sentence.

Among the search-on configurations, citation accuracy ranged from 37.1% (1,045 of 2,820 checked links) for Perplexity Sonar (via OpenRouter) to 91.4% (1,152 of 1,260 checked links) for Anthropic Claude Sonnet 5.5. OpenAI GPT-6 Sol came in at 43.5% (117 of 269 checked links) and Google Gemini 3.8 Flash with Google Search grounding on at 57.5% (506 of 880 checked links).

Dead links were a small part of the misses. For Sonar, 81 of 2,820 checked links were dead, while 1,694 of 2,820 led to a live page that did not support the sentence. For Claude Sonnet 5.5 with search on, the counts were 13 of 1,260 dead and 95 of 1,260 unsupported.

Links were checked with a plain HTTP request in the run week. Pages that load part of their content with scripts were judged on the text that request returned, so a few supporting passages may have been missed. This measure checks every cited link on any site, not only links to the brand, so it is not the same as the split into source, fact and attribution accuracy described in what structured data can and cannot do for the accuracy of AI citations.

A cited link shows what an answer displayed next to a sentence. It is not proof that the linked page says the same thing. When an answer cites your site, read the sentence next to the link as well as the link itself.

How consistent were repeated runs?

Each question went to each configuration three times. 81.1% of question and configuration pairs (284 of 350 pairs) gave the same outcome in all three runs, and the mean consistency score per configuration ranged from 0.87 to 0.95.

With three runs per question, this shows variation in this sample, not a long-run rate. It is still a reason to save more than one answer before acting on a single response, and to compare runs on the same questions over time. When the change you care about is a link to your own page that appears in one run and not the next, see how to measure links that disappear between runs.

What the findings mean for your brand

• Check the short facts answers repeat about you. Free plans, trials, user limits, plan names and headquarters are the kind of facts that answers state plainly and buyers act on. Read what answers say about each one.
• Keep one current version of each fact on your own pages. Put plan names, free-plan terms and limits on one pricing page, and describe your headquarters the same way across your site and investor pages. The guide to how regulated brands keep entity facts consistent applies the same discipline to health care. A clear page gives answers and readers one version to find; it does not guarantee that an answer will repeat it.
• Read the sentence next to each link. A citation to your site can sit next to a claim your page does not make.
• Save answers and compare them over time. One answer is one observation. The same questions asked again next month show whether a wrong fact persists.

Prime AI Visibility saves the answers to the questions you choose, with the brands they named and the links they cited, for ChatGPT, Claude, Perplexity, Gemini and an AI Overview-style preview built from Google-grounded answers. Its reports keep Brand named, Website cited and Source links separate, and checking a stated fact against your own page is a reading step you do with the saved answer. The features page shows the saved-answer evidence, the metrics and methodology page defines each measure, and how a check runs explains the steps from question to report.

When the fix is on your own pages and the sources that describe you, Percepture offers generative engine optimization services, and Prime AI Visibility has a commercial relationship with Percepture.

How to run the same check on your own brand

The benchmark's method works at a smaller scale.

1. Write down the facts buyers ask about. Free plan or trial, plan names, prices, user limits and headquarters, each with the page it comes from and the date you read it.
2. Ask the questions buyers ask. Use their wording, on the AI tools they use, and ask each question more than once.
3. Mark each stated fact. Matching, contradicting or not checkable against your page. Keep the counts as well as the percentage, so a small sample is visible as one.
4. Open the cited links. Note whether each linked page supports the sentence it sits next to.
5. Repeat on the same questions. Compare like with like, on the same tools and settings.

Prime AI Visibility can ask the questions and repeat them on a schedule, keeping each saved answer with its cited links; marking the facts against your own pages stays a reading job.

To compare your brand with competitors on the same questions, follow the step-by-step method for a fair competitor benchmark. The benchmark page publishes its full question set and a CSV with one row per answer, so you can reuse its questions, and the plan and credit table shows what scheduled checks include.

What Edition 1 does not show

• It is not a ranking of AI tools. It covers one question set, one market and one week, and answers change over time and by location.
• It does not cover consumer apps. The answers came from each provider's API. They are not captures of the ChatGPT, Claude, Gemini or Perplexity apps, of Google AI Overviews or of Google AI Mode.
• It does not explain why an answer was wrong. It records what each answer said and whether the brand's own page agreed.
• It does not describe smaller brands. The panel is 25 well-known brands. Leading brands that sell AI-visibility products were left out of the panel.
• AI models did the grading, with checks. An extractor, two graders from different providers and a support judge read the claims and links. Disagreements were checked against the first-party snapshot and every agreed error was re-read, but errors that both graders missed in the same way can remain.