Published study and dataset

Prime AI Visibility Benchmark

Edition 1 · October 23, 2026

50 buyer questions about 25 brands in five categories, sent three times each to seven AI model configurations through their APIs. Every answer was checked for the brands it named, whether its cited links supported it, whether its claims matched the brands' own pages, and whether repeated runs agreed.

50 questions · 7 model configurations · 1,050 answers collected

models tested with search on and off had a lower factual error rate with search on
3 of 3
OpenAI GPT-6 Sol: 3.3% on, 5.9% off; Anthropic Claude Sonnet 5.5: 8.4% on, 11.6% off; Google Gemini 3.8 Flash: 11.9% on, 18.1% off
of checked cited links supported the sentence they were attached to, depending on the search-on configuration
37.1%–91.4%
Perplexity Sonar (via OpenRouter): 1,045 of 2,820 checked links; Anthropic Claude Sonnet 5.5: 1,152 of 1,260 checked links
of checkable brand claims contradicted the brand's own pages
10%
183 of 1,821 checkable claims across all seven configurations
of question and configuration pairs gave the same outcome in all 3 runs
81.1%
284 of 350 pairs

01Timing tower

Every configuration against the four measures

Each row is one model configuration, measured on its usable answers. Sort by a measure to compare the values side by side. The tower shows what was measured in this edition; it is not a ranking.

Edition 1 · 7 configurations · 4 measures

Sorted by Run order.

Prime AI Visibility Benchmark, Edition 1: measured values per model configuration. Sorted by Run order.
ConfigurationBrand mentionsCitation accuracyFactual error rateConsistency
GPT-6 SolOpenAI GPT-6 Solsearch on95.6%43 of 45 discovery answers43.5%117 of 269 checked links3.3%9 of 273 checkable claims0.9443 of 50 questions alike in all 3 runs
GPT-6 SolOpenAI GPT-6 Solsearch off100%45 of 45 discovery answersNo linksNo cited links5.9%10 of 169 checkable claims0.9441 of 50 questions alike in all 3 runs
Claude Sonnet 5.5Anthropic Claude Sonnet 5.5search on100%45 of 45 discovery answers91.4%1,152 of 1,260 checked links8.4%23 of 275 checkable claims0.9545 of 50 questions alike in all 3 runs
Claude Sonnet 5.5Anthropic Claude Sonnet 5.5search off100%45 of 45 discovery answersNo linksNo cited links11.6%33 of 285 checkable claims0.9341 of 50 questions alike in all 3 runs
Gemini 3.8 FlashGoogle Gemini 3.8 Flashgrounding on100%45 of 45 discovery answers57.5%506 of 880 checked links11.9%38 of 318 checkable claims0.9139 of 50 questions alike in all 3 runs
Gemini 3.8 FlashGoogle Gemini 3.8 Flashgrounding off100%45 of 45 discovery answers0%0 of 1 checked link18.1%55 of 304 checkable claims0.9240 of 50 questions alike in all 3 runs
SonarPerplexity Sonar (via OpenRouter)search always on100%45 of 45 discovery answers37.1%1,045 of 2,820 checked links7.6%15 of 197 checkable claims0.8735 of 50 questions alike in all 3 runs
Brand mentions
Discovery answers that name at least one panel brand from the question's category.
Citation accuracy
Checked cited links that resolve to a live page and support the sentence they are attached to.
Factual error rate
Checkable claims that contradict the dated fact sheet of the brand's own pages.
Consistency
The most common outcome's share of a question's runs, averaged over the questions (0 to 1).

02By category

Category by configuration

The same four measures split by product category. Pick the measure to emphasise; every cell still carries all four values.

0%26.2%: colour runs from the lowest to the highest factual error rate in this view

Factual error rate: Checkable claims that contradict the dated fact sheet of the brand's own pages. Colour shows the value within this view, from the lowest to the highest; it is not a score.

Factual error rate by category and configuration (emphasised). Every cell also lists the other three measures.
ConfigurationCRM softwarePayroll softwareProject management softwareOnline travel bookingEcommerce platforms
GPT-6 Solsearch on
Factual error rate: 17.6%Mentions: 100%Citation accuracy: 45.9%Consistency: 1.00
Factual error rate: 0%Mentions: 100%Citation accuracy: 47.4%Consistency: 0.90
Factual error rate: 0%Mentions: 100%Citation accuracy: 35.3%Consistency: 0.97
Factual error rate: 0%Mentions: 88.9%Citation accuracy: 55.8%Consistency: 0.93
Factual error rate: 0%Mentions: 88.9%Citation accuracy: 39%Consistency: 0.90
GPT-6 Solsearch off
Factual error rate: 12.9%Mentions: 100%Citation accuracy: No linksConsistency: 1.00
Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.90
Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.97
Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.93
Factual error rate: 16.7%Mentions: 100%Citation accuracy: No linksConsistency: 0.90
Claude Sonnet 5.5search on
Factual error rate: 20%Mentions: 100%Citation accuracy: 91.1%Consistency: 0.93
Factual error rate: 0%Mentions: 100%Citation accuracy: 92.5%Consistency: 0.93
Factual error rate: 4.1%Mentions: 100%Citation accuracy: 90.8%Consistency: 0.97
Factual error rate: 0%Mentions: 100%Citation accuracy: 93.6%Consistency: 0.93
Factual error rate: 14.5%Mentions: 100%Citation accuracy: 90.5%Consistency: 1.00
Claude Sonnet 5.5search off
Factual error rate: 23.9%Mentions: 100%Citation accuracy: No linksConsistency: 0.93
Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.87
Factual error rate: 11.1%Mentions: 100%Citation accuracy: No linksConsistency: 0.97
Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.93
Factual error rate: 21.7%Mentions: 100%Citation accuracy: No linksConsistency: 0.97
Gemini 3.8 Flashgrounding on
Factual error rate: 16.4%Mentions: 100%Citation accuracy: 55.6%Consistency: 0.90
Factual error rate: 0%Mentions: 100%Citation accuracy: 64.9%Consistency: 0.90
Factual error rate: 13.2%Mentions: 100%Citation accuracy: 69.6%Consistency: 0.97
Factual error rate: 10%Mentions: 100%Citation accuracy: 75.6%Consistency: 0.90
Factual error rate: 18.6%Mentions: 100%Citation accuracy: 49.3%Consistency: 0.90
Gemini 3.8 Flashgrounding off
Factual error rate: 26.2%Mentions: 100%Citation accuracy: No linksConsistency: 0.87
Factual error rate: 0%Mentions: 100%Citation accuracy: 0%Consistency: 0.93
Factual error rate: 16.7%Mentions: 100%Citation accuracy: No linksConsistency: 0.97
Factual error rate: 14.8%Mentions: 100%Citation accuracy: No linksConsistency: 0.90
Factual error rate: 25.7%Mentions: 100%Citation accuracy: No linksConsistency: 0.93
Sonarsearch always on
Factual error rate: 15%Mentions: 100%Citation accuracy: 43%Consistency: 0.87
Factual error rate: 0%Mentions: 100%Citation accuracy: 35.8%Consistency: 0.87
Factual error rate: 2.1%Mentions: 100%Citation accuracy: 36.7%Consistency: 0.97
Factual error rate: 0%Mentions: 100%Citation accuracy: 30.8%Consistency: 0.87
Factual error rate: 18.6%Mentions: 100%Citation accuracy: 37.5%Consistency: 0.80

Columns: CRM = CRM software · Payroll = Payroll software · Project = Project management software · Travel = Online travel booking · Ecommerce = Ecommerce platforms. The other three measures for each cell are in the full table under this chart and in the data download.

Show every category as a table
CRM software
ConfigurationBrand mentionsCitation accuracyFactual error rateConsistency
OpenAI GPT-6 Sol — web search on100%9 of 9 discovery answers45.9%28 of 61 checked links17.6%9 of 51 checkable claims1.0010 of 10 questions alike in all 3 runs
OpenAI GPT-6 Sol — web search off100%9 of 9 discovery answersNo linksNo cited links12.9%4 of 31 checkable claims1.0010 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search on100%9 of 9 discovery answers91.1%288 of 316 checked links20%11 of 55 checkable claims0.939 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search off100%9 of 9 discovery answersNo linksNo cited links23.9%11 of 46 checkable claims0.938 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — Google Search grounding on100%9 of 9 discovery answers55.6%183 of 329 checked links16.4%10 of 61 checkable claims0.907 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — grounding off100%9 of 9 discovery answersNo linksNo cited links26.2%17 of 65 checkable claims0.877 of 10 questions alike in all 3 runs
Perplexity Sonar (via OpenRouter) — search always on100%9 of 9 discovery answers43%278 of 647 checked links15%6 of 40 checkable claims0.877 of 10 questions alike in all 3 runs
Payroll software
ConfigurationBrand mentionsCitation accuracyFactual error rateConsistency
OpenAI GPT-6 Sol — web search on100%9 of 9 discovery answers47.4%18 of 38 checked links0%0 of 42 checkable claims0.908 of 10 questions alike in all 3 runs
OpenAI GPT-6 Sol — web search off100%9 of 9 discovery answersNo linksNo cited links0%0 of 30 checkable claims0.907 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search on100%9 of 9 discovery answers92.5%185 of 200 checked links0%0 of 38 checkable claims0.938 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search off100%9 of 9 discovery answersNo linksNo cited links0%0 of 46 checkable claims0.877 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — Google Search grounding on100%9 of 9 discovery answers64.9%48 of 74 checked links0%0 of 61 checkable claims0.907 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — grounding off100%9 of 9 discovery answers0%0 of 1 checked link0%0 of 43 checkable claims0.939 of 10 questions alike in all 3 runs
Perplexity Sonar (via OpenRouter) — search always on100%9 of 9 discovery answers35.8%120 of 335 checked links0%0 of 30 checkable claims0.877 of 10 questions alike in all 3 runs
Project management software
ConfigurationBrand mentionsCitation accuracyFactual error rateConsistency
OpenAI GPT-6 Sol — web search on100%9 of 9 discovery answers35.3%24 of 68 checked links0%0 of 81 checkable claims0.979 of 10 questions alike in all 3 runs
OpenAI GPT-6 Sol — web search off100%9 of 9 discovery answersNo linksNo cited links0%0 of 42 checkable claims0.979 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search on100%9 of 9 discovery answers90.8%277 of 305 checked links4.1%3 of 73 checkable claims0.979 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search off100%9 of 9 discovery answersNo linksNo cited links11.1%9 of 81 checkable claims0.979 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — Google Search grounding on100%9 of 9 discovery answers69.6%96 of 138 checked links13.2%10 of 76 checkable claims0.979 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — grounding off100%9 of 9 discovery answersNo linksNo cited links16.7%12 of 72 checkable claims0.979 of 10 questions alike in all 3 runs
Perplexity Sonar (via OpenRouter) — search always on100%9 of 9 discovery answers36.7%229 of 624 checked links2.1%1 of 48 checkable claims0.979 of 10 questions alike in all 3 runs
Online travel booking
ConfigurationBrand mentionsCitation accuracyFactual error rateConsistency
OpenAI GPT-6 Sol — web search on88.9%8 of 9 discovery answers55.8%24 of 43 checked links0%0 of 30 checkable claims0.938 of 10 questions alike in all 3 runs
OpenAI GPT-6 Sol — web search off100%9 of 9 discovery answersNo linksNo cited links0%0 of 30 checkable claims0.938 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search on100%9 of 9 discovery answers93.6%146 of 156 checked links0%0 of 47 checkable claims0.939 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search off100%9 of 9 discovery answersNo linksNo cited links0%0 of 52 checkable claims0.938 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — Google Search grounding on100%9 of 9 discovery answers75.6%34 of 45 checked links10%5 of 50 checkable claims0.908 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — grounding off100%9 of 9 discovery answersNo linksNo cited links14.8%8 of 54 checkable claims0.907 of 10 questions alike in all 3 runs
Perplexity Sonar (via OpenRouter) — search always on100%9 of 9 discovery answers30.8%173 of 561 checked links0%0 of 36 checkable claims0.877 of 10 questions alike in all 3 runs
Ecommerce platforms
ConfigurationBrand mentionsCitation accuracyFactual error rateConsistency
OpenAI GPT-6 Sol — web search on88.9%8 of 9 discovery answers39%23 of 59 checked links0%0 of 69 checkable claims0.908 of 10 questions alike in all 3 runs
OpenAI GPT-6 Sol — web search off100%9 of 9 discovery answersNo linksNo cited links16.7%6 of 36 checkable claims0.907 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search on100%9 of 9 discovery answers90.5%256 of 283 checked links14.5%9 of 62 checkable claims1.0010 of 10 questions alike in all 3 runs
Anthropic Claude Sonnet 5.5 — web search off100%9 of 9 discovery answersNo linksNo cited links21.7%13 of 60 checkable claims0.979 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — Google Search grounding on100%9 of 9 discovery answers49.3%145 of 294 checked links18.6%13 of 70 checkable claims0.908 of 10 questions alike in all 3 runs
Google Gemini 3.8 Flash — grounding off100%9 of 9 discovery answersNo linksNo cited links25.7%18 of 70 checkable claims0.938 of 10 questions alike in all 3 runs
Perplexity Sonar (via OpenRouter) — search always on100%9 of 9 discovery answers37.5%245 of 653 checked links18.6%8 of 43 checkable claims0.805 of 10 questions alike in all 3 runs

04Consistency

Repeated runs, lap by lap

Every question was sent 3 times to each configuration. Each lap is one question and each dot one run, grouped by category.

  • GPT-6 Solweb search on · mean consistency 0.94All 3 runs alike: 43 of 50 questions
  • GPT-6 Solweb search off · mean consistency 0.94All 3 runs alike: 41 of 50 questions
  • Claude Sonnet 5.5web search on · mean consistency 0.95All 3 runs alike: 45 of 50 questions
  • Claude Sonnet 5.5web search off · mean consistency 0.93All 3 runs alike: 41 of 50 questions
  • Gemini 3.8 FlashGoogle Search grounding on · mean consistency 0.91All 3 runs alike: 39 of 50 questions
  • Gemini 3.8 Flashgrounding off · mean consistency 0.92All 3 runs alike: 40 of 50 questions
  • Sonarsearch always on · mean consistency 0.87All 3 runs alike: 35 of 50 questions
Show the data as a table
Questions by how many runs gave the same outcome, per configuration
ConfigurationMean consistencyAll 3 runs alike2 of 3 runs alikeNo two runs alike
OpenAI GPT-6 Sol — web search on0.9443 of 50 questions5 of 50 questions2 of 50 questions
OpenAI GPT-6 Sol — web search off0.9441 of 50 questions9 of 50 questions0 of 50 questions
Anthropic Claude Sonnet 5.5 — web search on0.9545 of 50 questions3 of 50 questions2 of 50 questions
Anthropic Claude Sonnet 5.5 — web search off0.9341 of 50 questions8 of 50 questions1 of 50 questions
Google Gemini 3.8 Flash — Google Search grounding on0.9139 of 50 questions9 of 50 questions2 of 50 questions
Google Gemini 3.8 Flash — grounding off0.9240 of 50 questions8 of 50 questions2 of 50 questions
Perplexity Sonar (via OpenRouter) — search always on0.8735 of 50 questions11 of 50 questions4 of 50 questions

05Brand naming

Which panel brands the answers named

Discovery questions only; usable answers in the brand's own category. Counts are what the answers showed; they do not explain why a brand was or was not named.

CRM software

  • Salesforce
    Named in 39 of 63 answers
  • Microsoft Dynamics 365 Sales
    Named in 5 of 63 answers
  • Zoho CRM
    Named in 49 of 63 answers
  • Pipedrive
    Named in 43 of 63 answers
  • Freshsales
    Named in 24 of 63 answers
Counts per configuration
CRM software: answers naming each brand, per configuration
BrandGPT-6 Sol, search onGPT-6 Sol, search offClaude Sonnet 5.5, search onClaude Sonnet 5.5, search offGemini 3.8 Flash, grounding onGemini 3.8 Flash, grounding offSonar, search always on
Salesforce3 of 93 of 99 of 96 of 95 of 94 of 99 of 9
Microsoft Dynamics 365 Sales0 of 90 of 92 of 93 of 90 of 90 of 90 of 9
Zoho CRM9 of 96 of 98 of 98 of 96 of 96 of 96 of 9
Pipedrive6 of 96 of 96 of 98 of 95 of 96 of 96 of 9
Freshsales3 of 93 of 93 of 96 of 93 of 94 of 92 of 9

Payroll software

  • Gusto
    Named in 62 of 63 answers
  • ADP
    Named in 46 of 63 answers
  • Paychex
    Named in 17 of 63 answers
  • Rippling
    Named in 27 of 63 answers
  • QuickBooks Payroll
    Named in 34 of 63 answers
Counts per configuration
Payroll software: answers naming each brand, per configuration
BrandGPT-6 Sol, search onGPT-6 Sol, search offClaude Sonnet 5.5, search onClaude Sonnet 5.5, search offGemini 3.8 Flash, grounding onGemini 3.8 Flash, grounding offSonar, search always on
Gusto9 of 99 of 99 of 99 of 99 of 99 of 98 of 9
ADP3 of 95 of 99 of 99 of 98 of 95 of 97 of 9
Paychex0 of 90 of 98 of 96 of 90 of 90 of 93 of 9
Rippling2 of 92 of 93 of 94 of 96 of 96 of 94 of 9
QuickBooks Payroll3 of 96 of 96 of 97 of 94 of 95 of 93 of 9

Project management software

  • Asana
    Named in 63 of 63 answers
  • monday.com
    Named in 35 of 63 answers
  • Trello
    Named in 51 of 63 answers
  • ClickUp
    Named in 62 of 63 answers
  • Smartsheet
    Named in 2 of 63 answers
Counts per configuration
Project management software: answers naming each brand, per configuration
BrandGPT-6 Sol, search onGPT-6 Sol, search offClaude Sonnet 5.5, search onClaude Sonnet 5.5, search offGemini 3.8 Flash, grounding onGemini 3.8 Flash, grounding offSonar, search always on
Asana9 of 99 of 99 of 99 of 99 of 99 of 99 of 9
monday.com0 of 93 of 97 of 99 of 95 of 95 of 96 of 9
Trello6 of 96 of 99 of 99 of 96 of 96 of 99 of 9
ClickUp8 of 99 of 99 of 99 of 99 of 99 of 99 of 9
Smartsheet0 of 90 of 90 of 90 of 90 of 90 of 92 of 9

Online travel booking

  • Booking.com
    Named in 39 of 63 answers
  • Expedia
    Named in 38 of 63 answers
  • Priceline
    Named in 27 of 63 answers
  • Trip.com
    Named in 22 of 63 answers
  • Agoda
    Named in 35 of 63 answers
Counts per configuration
Online travel booking: answers naming each brand, per configuration
BrandGPT-6 Sol, search onGPT-6 Sol, search offClaude Sonnet 5.5, search onClaude Sonnet 5.5, search offGemini 3.8 Flash, grounding onGemini 3.8 Flash, grounding offSonar, search always on
Booking.com3 of 94 of 97 of 97 of 95 of 96 of 97 of 9
Expedia3 of 93 of 96 of 96 of 97 of 98 of 95 of 9
Priceline3 of 92 of 95 of 94 of 95 of 94 of 94 of 9
Trip.com3 of 93 of 93 of 93 of 93 of 93 of 94 of 9
Agoda3 of 93 of 96 of 95 of 96 of 96 of 96 of 9

Ecommerce platforms

  • Shopify
    Named in 55 of 63 answers
  • WooCommerce
    Named in 56 of 63 answers
  • BigCommerce
    Named in 33 of 63 answers
  • Ecwid
    Named in 18 of 63 answers
  • Square Online
    Named in 33 of 63 answers
Counts per configuration
Ecommerce platforms: answers naming each brand, per configuration
BrandGPT-6 Sol, search onGPT-6 Sol, search offClaude Sonnet 5.5, search onClaude Sonnet 5.5, search offGemini 3.8 Flash, grounding onGemini 3.8 Flash, grounding offSonar, search always on
Shopify7 of 97 of 99 of 99 of 98 of 98 of 97 of 9
WooCommerce6 of 96 of 99 of 99 of 99 of 99 of 98 of 9
BigCommerce3 of 94 of 96 of 96 of 94 of 94 of 96 of 9
Ecwid0 of 90 of 96 of 93 of 92 of 93 of 94 of 9
Square Online1 of 95 of 96 of 96 of 95 of 95 of 95 of 9

06Annotated examples

What the checks look like on real answers

Excerpts from collected answers, each beside the fact-sheet value and the brand's own page it was checked against.

Accurate citationRun 1

Anthropic Claude Sonnet 5.5 — web search on

Question Does Zoho CRM have a free plan, and how many users can use it?

“Zoho CRM has a free edition that supports up to 3 users.”

Cited link https://www.zoho.com/en-us/crm/zohocrm-pricing.htmlLive page (HTTP 200) that supports the sentence

Fact sheet · Zoho CRM · Free plan
Yes — free edition for up to 3 users
First-party source · fact sheet dated October 23, 2026
https://www.zoho.com/crm/zohocrm-pricing.html
Grading
Matches the fact sheet (both graders agreed)
Broken linkRun 3

OpenAI GPT-6 Sol — web search on

Question Does Zoho CRM have a free plan, and how many users can use it?

“Zoho CRM has a free plan for up to 3 users.”

Cited link https://www.zoho.com/sites/default/files/bigin/bigin-zohocrm-editions-comparison.pdf?utm_source=openaiDead link: HTTP 404 when checked in the run week

Fact sheet · Zoho CRM · Free plan
Yes — free edition for up to 3 users
First-party source · fact sheet dated October 23, 2026
https://www.zoho.com/crm/zohocrm-pricing.html
Grading
Matches the fact sheet (both graders agreed)
Factual errorRun 1

OpenAI GPT-6 Sol — web search off

Question What plans does BigCommerce offer, and does it have a free plan?

“BigCommerce offers four plans: Standard, Plus, Pro, and Enterprise”
Fact sheet · BigCommerce · Plan names
Core, Growth, Scale, Performance
First-party source · fact sheet dated October 23, 2026
https://www.bigcommerce.com/essentials/pricing/
Grading
Contradicts the fact sheet (both graders agreed)

07Methodology

Method and run conditions

Scope. These are answers from each provider's API, not consumer-app sessions, and they are not live Google AI Overviews or Google AI Mode. Google appears here only as Gemini, with Google Search grounding switched on or off.

Run conditions

Prime AI Visibility Benchmark, Edition 1: run conditions per model configuration
ConfigurationProvider and APIRequested modelReturned versionSearchAnswers collectedRuns
OpenAI GPT-6 Solsearch onOpenAI · Responses APIgpt-6-solgpt-6-solOpenAI web_search toolSearched in 150 of 150 returned answersOctober 23, 2026, 11:22 to 11:53 UTC3
OpenAI GPT-6 Solsearch offOpenAI · Responses APIgpt-6-solgpt-6-solNoneOctober 23, 2026, 11:22 to 11:57 UTC3
Anthropic Claude Sonnet 5.5search onAnthropic · Messages APIclaude-sonnet-5-5claude-sonnet-5-5Anthropic web_search tool (web_search_20250305)Searched in 130 of 150 returned answersOctober 23, 2026, 11:22 to 11:52 UTC3
Anthropic Claude Sonnet 5.5search offAnthropic · Messages APIclaude-sonnet-5-5claude-sonnet-5-5NoneOctober 23, 2026, 11:52 to 11:58 UTC3
Google Gemini 3.8 Flashgrounding onGoogle · Gemini API generateContentgemini-3.8-flashgemini-3.8-flashGrounding with Google Search (googleSearch tool)Searched in 64 of 150 returned answersOctober 23, 2026, 11:23 to 12:01 UTC3
Google Gemini 3.8 Flashgrounding offGoogle · Gemini API generateContentgemini-3.8-flashgemini-3.8-flashNoneOctober 23, 2026, 12:01 to 12:21 UTC3
Perplexity Sonar (via OpenRouter)search always onPerplexity · OpenRouter Chat Completionsperplexity/sonarperplexity/sonarSonar's built-in web search (cannot be switched off)October 23, 2026, 11:23 to 11:40 UTC3

Prompting. Each question is sent exactly as written in the question set, as a single user message. No system prompt, no instructions, no conversation history. Nothing else is added.

Market. Questions are written in US English. Where an API accepts a location hint for its search tool, the hint is country US; otherwise no location is sent.

Sampling settings per configuration
Sampling settings per configuration
ConfigurationSampling
OpenAI GPT-6 Sol — web search onProvider defaults: temperature, top_p and reasoning effort are not sent.
OpenAI GPT-6 Sol — web search offProvider defaults: temperature, top_p and reasoning effort are not sent.
Anthropic Claude Sonnet 5.5 — web search onProvider defaults: temperature and top_p are not sent; extended thinking is not enabled.
Anthropic Claude Sonnet 5.5 — web search offProvider defaults: temperature and top_p are not sent; extended thinking is not enabled.
Google Gemini 3.8 Flash — Google Search grounding onProvider defaults: temperature, topP and thinking settings are not sent. The googleSearch tool takes no location parameter, so no location hint is sent.
Google Gemini 3.8 Flash — grounding offProvider defaults: temperature, topP and thinking settings are not sent.
Perplexity Sonar (via OpenRouter) — search always onProvider defaults: temperature, top_p and search options are not sent. Sonar has no search-off mode, so this configuration has no search-off pair.

Answers requested, returned, usable and missing

Requested
1,050
Returned
1,050
Usable
1,050
Missing
0
Answer counts per configuration
Answers requested, returned, usable and missing per configuration
ConfigurationRequestedReturnedUsableMissing
OpenAI GPT-6 Sol — web search on1501501500
OpenAI GPT-6 Sol — web search off1501501500
Anthropic Claude Sonnet 5.5 — web search on1501501500
Anthropic Claude Sonnet 5.5 — web search off1501501500
Google Gemini 3.8 Flash — Google Search grounding on1501501500
Google Gemini 3.8 Flash — grounding off1501501500
Perplexity Sonar (via OpenRouter) — search always on1501501500

Definitions

Brand named
Answers that explicitly name the brand or an approved name variant.
Consistency
Consistency = most-common outcome count ÷ number of runs
Fact accuracy
What the answer says about you matches what your page says: name, price, dates, location, product limits.
Source accuracy
The link under the answer goes to the right page on your site, not an old URL, a copy or a third-party page.
Citation accuracy
Benchmark measure: the share of cited links that resolve to a live page and support the claim they are attached to. Dead links, links that redirect to a homepage and unsupported links are counted separately; links that could not be checked are reported separately and left out of the denominator. A link attached to several sentences is counted once for each sentence. It checks every cited link on any site, so it is not the same as the Journal's split of "citation accuracy" into source, fact and attribution accuracy.
Citation accuracy = supported links ÷ (supported + unsupported + dead + redirected-to-homepage links)
Factual error rate
Benchmark measure: checkable claims that contradict the dated fact sheet ÷ checkable claims. A claim is checkable when the fact sheet covers it. Two grader models grade every claim; disagreements are checked against the brand's first-party source. Unresolved claims are left out of the rate and counted.
Factual error rate = claims that contradict the fact sheet ÷ checkable claims (unresolved claims excluded and counted)
Requested / returned / usable / missing
Requested: answers the edition asked for (questions × runs × configurations). Returned: the API returned an answer with text. Usable: a returned answer that responds to the question (refusals, clarification requests and off-topic replies are returned but not usable). Missing: the run produced no answer text (failed call, timeout or empty response). Percentages use the stated denominator.

Question types and what counts as the same outcome

Discovery15 questions
The set of this category's panel brands the answer names (approved variants count; order and wording ignored). Two runs share an outcome when they name exactly the same set of panel brands, including the empty set.
Brand fact25 questions
For each fact field the question asks about, the answer's resolved verdict against the fact sheet: consistent, contradicts, not stated or unresolved. Two runs share an outcome when every asked field has the same verdict.
Head to head10 questions
The pick the answer states: the first brand, the second brand, or no single pick (the answer declines to choose or says it depends). Two runs share an outcome when they state the same pick.

How scoring and grading work

Scoring models and their roles
RoleModelWhat it does
Extractorgemini-3.8-flashClassifies usability, extracts checkable fact claims and the head-to-head pick from each answer.
Grader Agpt-6-lunaGrader A for every checkable claim against the fact sheet.
Grader Bclaude-haiku-5-5Grader B for every checkable claim against the fact sheet.
Support judgegemini-3.8-flashJudges whether a live cited page supports the answer text the link is attached to.
  • 1,914 fact claims extracted; 1,821 were checkable against the fact sheet.
  • 82 grader disagreements, 82 resolved against the brand's first-party source snapshot, 0 unresolved.
  • 38 verdicts both graders agreed on were changed in the sanity check (37 to not checkable, 1 to consistent).

Disagreements between the two graders were checked against the brand's first-party source snapshot and recorded in a private review sheet. The sanity check also re-read every claim both graders marked as an error; verdicts it changed are counted here and logged in the same sheet.

Scoring calls are batched because the integrations proxy rate-limits: the extractor reads the runs of one (configuration, question) together, graders take up to 25 claims per call, the support judge up to 6 (page, claim) pairs per call.

Scoring ran October 23, 2026, 11:37 to 13:57 UTC.

The fact sheet

Dated October 23, 2026. 55 facts about 25 brands, from 38 snapshots of the brands' own pages taken October 23, 2026, 11:06 to 11:13 UTC.

A claim is an error only when it contradicts the fact as stated here. A claim that is incomplete but compatible, or about something this sheet does not cover, is not an error (it is either consistent or not checkable).

Show every fact and its source
CRM software
BrandFieldFactSource
SalesforceFounded1999https://www.salesforce.com/company/about-us/
SalesforceFree planYes — Free Suite, $0, comes with 2 user licenseshttps://www.salesforce.com/crm/free-crm/
SalesforceHeadquartersSan Francisco, Californiahttps://www.salesforce.com/company/locations/
Microsoft Dynamics 365 SalesFree planNo — free trial onlyhttps://www.microsoft.com/en-us/dynamics-365/products/sales/pricing
Microsoft Dynamics 365 SalesPlan namesSales Professional, Sales Enterprise, Sales Premium, Microsoft Relationship Saleshttps://www.microsoft.com/en-us/dynamics-365/products/sales/pricing
Zoho CRMFree planYes — free edition for up to 3 usershttps://www.zoho.com/crm/zohocrm-pricing.html
PipedriveFounded2010https://www.pipedrive.com/en/about
PipedriveHeadquartersNew York, NYhttps://www.pipedrive.com/en/about
PipedriveFree planNo — 14-day free trial; paid plans Lite, Growth, Premium, Ultimatehttps://www.pipedrive.com/en/pricing
FreshsalesFree planNo — 21-day free trial, then a paid planhttps://www.freshworks.com/crm/pricing/
FreshsalesPlan namesGrowth, Pro, Enterprisehttps://www.freshworks.com/crm/pricing/
FreshsalesHeadquartersSan Mateo, California (Freshworks)https://ir.freshworks.com/resources/investor-faqs/default.aspx
Payroll software
BrandFieldFactSource
GustoPlan namesSimple, Plus, Premium (plus a separate plan for paying yourself)https://gusto.com/product/pricing
GustoFormer nameZenPayroll (launched in 2012)https://gusto.com/about
ADPHeadquartersRoseland, New Jerseyhttps://mediacenter.adp.com/image/ADP+Corporate+Fact+Sheet.pdf
ADPFounded1949https://mediacenter.adp.com/image/ADP+Corporate+Fact+Sheet.pdf
PaychexHeadquartersDual headquarters: Rochester, NY and Cincinnati, OHhttps://www.paychex.com/corporate
PaychexFounded1971https://www.paychex.com/corporate
RipplingHeadquartersSan Francisco, Californiahttps://www.rippling.com/press
RipplingCEOParker Conrad (co-founder and CEO)https://www.rippling.com/about
QuickBooks PayrollParent companyIntuithttps://www.intuit.com/company/
QuickBooks PayrollRelated productsTurboTax, Credit Karma, Mailchimp, Intuit Enterprise Suitehttps://www.intuit.com/company/
Project management software
BrandFieldFactSource
AsanaFree planYes — Personal plan, $0, for one or two peoplehttps://asana.com/pricing
AsanaHeadquartersSan Francisco, Californiahttps://asana.com/company
AsanaPlan namesPersonal, Starter, Advanced, Enterprise (Enterprise+ for advanced security)https://asana.com/pricing
monday.comFree planYes — Free plan, $0, up to 2 seatshttps://monday.com/pricing
monday.comPlan namesFree, Basic, Standard, Pro, Enterprisehttps://monday.com/pricing
TrelloFree planYes — Free, $0, up to 10 collaborators per Workspacehttps://trello.com/pricing
TrelloParent companyAtlassian (acquired early 2017)https://trello.com/about
TrelloLaunch year2011https://trello.com/about
ClickUpHeadquartersSan Diego, Californiahttps://clickup.com/about
ClickUpFree planYes — Free Forever planhttps://clickup.com/pricing
SmartsheetFounded2005https://www.smartsheet.com/about
SmartsheetHeadquartersBellevue, Washingtonhttps://www.smartsheet.com/about
Online travel booking
BrandFieldFactSource
Booking.comHeadquartersAmsterdam, the Netherlandshttps://www.booking.com/content/about.html
Booking.comParent companyBooking Holdings Inc.https://www.booking.com/content/about.html
Booking.comFounded1996https://www.booking.com/content/about.html
ExpediaFounded1996https://www.expediagroup.com/en-us/about-us
ExpediaOriginStarted inside Microsofthttps://www.expediagroup.com/en-us/about-us
PricelineLaunch year1998https://press.priceline.com/about-priceline
PricelineParent companyBooking Holdingshttps://www.bookingholdings.com/about/
Trip.comFounded1999https://group.trip.com/about/story?locale=en-US
Trip.comFormer nameCtrip (www.ctrip.com)https://group.trip.com/about/story?locale=en-US
AgodaHeadquartersSingaporehttps://www.agoda.com/info/about-agoda.html
AgodaParent companyBooking Holdingshttps://www.agoda.com/info/about-agoda.html
Ecommerce platforms
BrandFieldFactSource
ShopifyPlan namesBasic, Grow, Advanced, Plushttps://www.shopify.com/pricing
ShopifyHeadquartersLas Vegas, Nevada (principal executive office)https://shopify.com/investors/investor-faqs
WooCommerceFree planYes — the core platform is free and open sourcehttps://woocommerce.com/about/
WooCommerceParent companyWoo, part of Automattic (acquired 2015)https://woocommerce.com/about/
BigCommercePlan namesCore, Growth, Scale, Performancehttps://www.bigcommerce.com/essentials/pricing/
BigCommerceFree planNo — free trial onlyhttps://www.bigcommerce.com/essentials/pricing/
EcwidFree planNo — plans are Starter, Venture, Business and Unlimited, all paidhttps://www.ecwid.com/pricing
EcwidParent companyLightspeedhttps://support.ecwid.com/hc/en-us/articles/5582039267996-Ecwid-is-now-Lightspeed
Square OnlineFree planYes — Square Free, $0/month per locationhttps://squareup.com/us/en/online-store/plans
Square OnlinePlan namesSquare Free, Square Plus, Square Premiumhttps://squareup.com/us/en/online-store/plans

Limitations

  • API answers only. These are not captures of the ChatGPT, Claude, Gemini or Perplexity apps, of Google AI Overviews or of Google AI Mode, and consumer apps can answer differently.
  • One market and language (United States, US English) and one collection week. Answers change over time and can differ by location.
  • Three runs per question per configuration, so a question's consistency can only be 1/3, 2/3 or 1. It shows variation in this sample, not a long-run rate.
  • Each question was sent as written with provider default settings and no system prompt. Other settings, prompts or follow-up questions can produce different answers.
  • The factual error rate covers only facts in the dated fact sheet, taken from each brand's own pages in the run week. A claim counts as an error when it contradicts the brand's own page as snapshotted; statements on topics outside the fact sheet were not graded.
  • Each fact is checked against the page named as its source. Where a brand describes itself differently elsewhere (for example a company that names one principal executive office but is widely described as based in another city), answers that repeat the other description count as contradicting the named page. Plans or offices that the source page does not list were treated as not checkable rather than as errors.
  • Claims, picks and citation support were read by AI models (an extractor, two graders from different providers and a support judge). Grader disagreements were checked against the first-party snapshot, and every claim both graders marked as an error was re-read in a sanity check; errors that both graders missed in the same way can remain.
  • Cited links were checked with a plain HTTP request (no browser) in the run week. Pages that blocked automated requests, returned too little readable text without a browser, or were PDFs are counted as unverifiable and left out of the citation-accuracy denominator. Pages that load part of their content with scripts were judged on the text the plain request returned, so a few supporting passages may have been missed.
  • With Google Search grounding on, Gemini decides whether to search; some grounding-on answers ran no search at all and are still counted in that configuration.
  • Perplexity Sonar was reached through OpenRouter rather than Perplexity's own API; Sonar always searches, so it has no search-off pair.
  • The panel is 25 well-known brands in five categories. Leading brands that sell AI-visibility products (for example HubSpot in CRM) were left out, so answers that name them are not scored for them. Results do not describe smaller or lesser-known brands.

Not included

Perplexity Sonar with a direct Perplexity API key
Not needed: Sonar was reachable through the Replit-managed OpenRouter route, so no key was requested. OpenRouter forwards to Perplexity; answers may differ slightly from Perplexity's own API.
Consumer apps (ChatGPT app, Claude app, Gemini app, Google AI Overviews, Google AI Mode, Perplexity app)
Out of scope. The benchmark measures API answers only; they are not captures of any consumer app or search results page.
Search-off for Perplexity Sonar
Sonar always searches; the API offers no search-off mode.

Comparability. First edition: there is no previous edition to compare against.

08Question set

The full question set

Show all 50 questionsQuestion set v1, grouped by category, exactly as sent

CRM software

  1. DiscoveryWhat is the best CRM for a small business?crm-discovery-1
  2. DiscoveryWhich CRM should a B2B sales team of about 20 people use?crm-discovery-2
  3. DiscoveryWhat are the best CRM tools with a free plan?crm-discovery-3
  4. Brand factWhen was Salesforce founded, and does Salesforce offer a free CRM plan?Salesforce · Checks: Founded, Free plancrm-fact-salesforce
  5. Brand factDoes Microsoft Dynamics 365 Sales have a free plan, and what plans does it offer?Microsoft Dynamics 365 Sales · Checks: Free plan, Plan namescrm-fact-dynamics-365-sales
  6. Brand factDoes Zoho CRM have a free plan, and how many users can use it?Zoho CRM · Checks: Free plancrm-fact-zoho-crm
  7. Brand factWhen was Pipedrive founded, and where is it headquartered?Pipedrive · Checks: Founded, Headquarterscrm-fact-pipedrive
  8. Brand factDoes Freshsales have a free plan, and what paid plans does it offer?Freshsales · Checks: Free plan, Plan namescrm-fact-freshsales
  9. Head to headSalesforce or Zoho CRM: which is better for a growing small business?Salesforce vs Zoho CRMcrm-h2h-1
  10. Head to headPipedrive vs Freshsales: which CRM is better for a small sales team?Pipedrive vs Freshsalescrm-h2h-2

Payroll software

  1. DiscoveryWhat is the best payroll software for a small business?payroll-discovery-1
  2. DiscoveryWhich payroll service is best for a company with employees in several US states?payroll-discovery-2
  3. DiscoveryWhat is the easiest payroll software for a business with fewer than 10 employees?payroll-discovery-3
  4. Brand factWhat plans does Gusto offer, and what was Gusto originally called?Gusto · Checks: Plan names, Former namepayroll-fact-gusto
  5. Brand factWhere is ADP headquartered, and what year was it founded?ADP · Checks: Headquarters, Foundedpayroll-fact-adp
  6. Brand factWhere is Paychex headquartered, and what year was it founded?Paychex · Checks: Headquarters, Foundedpayroll-fact-paychex
  7. Brand factWhere is Rippling based, and who is its CEO?Rippling · Checks: Headquarters, CEOpayroll-fact-rippling
  8. Brand factWhich company makes QuickBooks Payroll, and what other well-known products does that company make?QuickBooks Payroll · Checks: Parent company, Related productspayroll-fact-quickbooks-payroll
  9. Head to headGusto or ADP: which payroll provider is better for a small business?Gusto vs ADPpayroll-h2h-1
  10. Head to headRippling vs Paychex: which is better for a growing company?Rippling vs Paychexpayroll-h2h-2

Project management software

  1. DiscoveryWhat is the best project management software for a small team?pm-discovery-1
  2. DiscoveryWhich project management tool is best for a marketing agency?pm-discovery-2
  3. DiscoveryWhat are the best free project management tools?pm-discovery-3
  4. Brand factDoes Asana have a free plan, and where is Asana headquartered?Asana · Checks: Free plan, Headquarterspm-fact-asana
  5. Brand factDoes monday.com have a free plan, and what paid plans does it offer?monday.com · Checks: Free plan, Plan namespm-fact-monday
  6. Brand factDoes Trello have a free plan, and which company owns Trello?Trello · Checks: Free plan, Parent companypm-fact-trello
  7. Brand factWhere is ClickUp headquartered, and does it have a free plan?ClickUp · Checks: Headquarters, Free planpm-fact-clickup
  8. Brand factWhen was Smartsheet founded, and where is it headquartered?Smartsheet · Checks: Founded, Headquarterspm-fact-smartsheet
  9. Head to headAsana or monday.com: which is better for managing team projects?Asana vs monday.compm-h2h-1
  10. Head to headTrello vs ClickUp: which is better for a small team?Trello vs ClickUppm-h2h-2

Online travel booking

  1. DiscoveryWhat is the best website to book a hotel?travel-discovery-1
  2. DiscoveryWhich online travel site is best for booking flights and hotels together?travel-discovery-2
  3. DiscoveryWhat is the best travel booking site for trips in Asia?travel-discovery-3
  4. Brand factWhere is Booking.com based, and which company owns it?Booking.com · Checks: Headquarters, Parent companytravel-fact-booking-com
  5. Brand factWhat year did Expedia launch, and which company was it started inside?Expedia · Checks: Founded, Origintravel-fact-expedia
  6. Brand factWhen was Priceline launched, and which company owns Priceline today?Priceline · Checks: Launch year, Parent companytravel-fact-priceline
  7. Brand factWhat year was Trip.com Group founded, and what was its original website called?Trip.com · Checks: Founded, Former nametravel-fact-trip-com
  8. Brand factWhere is Agoda headquartered, and which company owns it?Agoda · Checks: Headquarters, Parent companytravel-fact-agoda
  9. Head to headBooking.com or Expedia: which is better for booking hotels?Booking.com vs Expediatravel-h2h-1
  10. Head to headTrip.com vs Agoda: which is better for booking travel in Asia?Trip.com vs Agodatravel-h2h-2

Ecommerce platforms

  1. DiscoveryWhat is the best ecommerce platform for a small business?ecom-discovery-1
  2. DiscoveryWhich ecommerce platform should I use to start selling online for free?ecom-discovery-2
  3. DiscoveryWhich ecommerce platform is best for a fast-growing online store?ecom-discovery-3
  4. Brand factWhat plans does Shopify offer, and where is Shopify's corporate headquarters?Shopify · Checks: Plan names, Headquartersecom-fact-shopify
  5. Brand factIs WooCommerce free to use, and which company is behind it?WooCommerce · Checks: Free plan, Parent companyecom-fact-woocommerce
  6. Brand factWhat plans does BigCommerce offer, and does it have a free plan?BigCommerce · Checks: Plan names, Free planecom-fact-bigcommerce
  7. Brand factDoes Ecwid have a free plan, and which company owns Ecwid?Ecwid · Checks: Free plan, Parent companyecom-fact-ecwid
  8. Brand factDoes Square Online have a free plan, and what are its plan names?Square Online · Checks: Free plan, Plan namesecom-fact-square-online
  9. Head to headShopify or WooCommerce: which is better for a small online store?Shopify vs WooCommerceecom-h2h-1
  10. Head to headBigCommerce vs Shopify: which is better for a growing business?BigCommerce vs Shopifyecom-h2h-2

09Data and citation

Download, cite and reuse

License. The data, the question set and the charts on this page are published under Creative Commons Attribution 4.0 International (CC BY 4.0). Reuse them freely; credit the Prime AI Visibility Benchmark, Edition 1 and link to this page.

Cite this benchmark

Prime AI Visibility. (2026). Prime AI Visibility Benchmark, Edition 1 [Data set]. https://primeaivisibility.com/benchmark

BibTeX

@misc{primeaivisibility_benchmark_2026_e1,
  title = {Prime AI Visibility Benchmark, Edition 1},
  author = {{Prime AI Visibility}},
  year = {2026},
  howpublished = {\url{https://primeaivisibility.com/benchmark}},
  note = {Data set, CC BY 4.0}
}

10Changelog and archive

Editions

Changelog

  1. Edition 1 · October 23, 2026

    • First edition.

Editions

  • Edition 1 (this page)October 23, 2026 · latest

The latest edition always lives at /benchmark. When a new edition is published, earlier editions keep their own permanent pages, linked here.

Run your own brand through questions like these

Flash is one free check each calendar month. Compare up to five buyer questions across ChatGPT, Claude, and Perplexity, with up to 15 saved answers and their available sources.

5 questions · 3 AI tools · Up to 15 answers · No card required

Building a competitor benchmark of your own? The method is in How to benchmark your brand's AI citations vs competitors.

Frequently asked questions

What is the Prime AI Visibility Benchmark?
A published study and dataset of how AI models answer buyer questions about brands. Edition 1 sent 50 questions about 25 brands in five categories to seven model configurations through their APIs, three times each, and scored 1,050 usable answers for brand mentions, citation accuracy, factual errors and consistency.
Which AI models and settings were tested?
OpenAI GPT-6 Sol — web search on; OpenAI GPT-6 Sol — web search off; Anthropic Claude Sonnet 5.5 — web search on; Anthropic Claude Sonnet 5.5 — web search off; Google Gemini 3.8 Flash — Google Search grounding on; Google Gemini 3.8 Flash — grounding off; Perplexity Sonar (via OpenRouter) — search always on. Each configuration used the provider's default settings with no system prompt; the run-conditions table lists the returned model version and collection times.
Does the benchmark measure the ChatGPT app, Google AI Overviews or AI Mode?
No. These are answers from each provider's API, not consumer-app sessions, and they are not live Google AI Overviews or Google AI Mode. Google appears here only as Gemini, with Google Search grounding switched on or off.
How is the factual error rate calculated?
Each checkable claim about a panel brand is compared with a dated fact sheet built from the brand's own pages. Two grader models from different providers grade every claim, and disagreements are checked against the first-party source. The rate is claims that contradict the fact sheet divided by checkable claims.
How is citation accuracy calculated?
Every cited link is checked with a plain HTTP request in the run week. A link counts as accurate when it resolves to a live page that supports the sentence it is attached to. Dead links, links that redirect to a homepage and unsupported links count against it; links that could not be checked are reported separately and left out.
Can I reuse the data?
Yes. The CSV, the question set and the charts are published under Creative Commons Attribution 4.0 International (CC BY 4.0), so you can reuse them with attribution. Cite it as: Prime AI Visibility. (2026). Prime AI Visibility Benchmark, Edition 1 [Data set]. https://primeaivisibility.com/benchmark
How can I see how AI answers describe my own brand?
Run a free Flash check in Prime AI Visibility: one check each calendar month that compares up to five of your buyer questions across ChatGPT, Claude, and Perplexity, with up to 15 saved answers and their available sources. To build a competitor benchmark of your own, read the Journal guide "How to benchmark your brand's AI citations vs competitors".