Published study and dataset
Prime AI Visibility Benchmark
Edition 1 · October 23, 2026
50 buyer questions about 25 brands in five categories, sent three times each to seven AI model configurations through their APIs. Every answer was checked for the brands it named, whether its cited links supported it, whether its claims matched the brands' own pages, and whether repeated runs agreed.
50 questions · 7 model configurations · 1,050 answers collected
- models tested with search on and off had a lower factual error rate with search on
- 3 of 3
- OpenAI GPT-6 Sol: 3.3% on, 5.9% off; Anthropic Claude Sonnet 5.5: 8.4% on, 11.6% off; Google Gemini 3.8 Flash: 11.9% on, 18.1% off
- of checked cited links supported the sentence they were attached to, depending on the search-on configuration
- 37.1%–91.4%
- Perplexity Sonar (via OpenRouter): 1,045 of 2,820 checked links; Anthropic Claude Sonnet 5.5: 1,152 of 1,260 checked links
- of checkable brand claims contradicted the brand's own pages
- 10%
- 183 of 1,821 checkable claims across all seven configurations
- of question and configuration pairs gave the same outcome in all 3 runs
- 81.1%
- 284 of 350 pairs
01Timing tower
Every configuration against the four measures
Each row is one model configuration, measured on its usable answers. Sort by a measure to compare the values side by side. The tower shows what was measured in this edition; it is not a ranking.
Edition 1 · 7 configurations · 4 measures
Sorted by Run order.
| Configuration | Brand mentions | Citation accuracy | Factual error rate | Consistency |
|---|---|---|---|---|
| GPT-6 SolOpenAI GPT-6 Solsearch on | 95.6%43 of 45 discovery answers | 43.5%117 of 269 checked links | 3.3%9 of 273 checkable claims | 0.9443 of 50 questions alike in all 3 runs |
| GPT-6 SolOpenAI GPT-6 Solsearch off | 100%45 of 45 discovery answers | No linksNo cited links | 5.9%10 of 169 checkable claims | 0.9441 of 50 questions alike in all 3 runs |
| Claude Sonnet 5.5Anthropic Claude Sonnet 5.5search on | 100%45 of 45 discovery answers | 91.4%1,152 of 1,260 checked links | 8.4%23 of 275 checkable claims | 0.9545 of 50 questions alike in all 3 runs |
| Claude Sonnet 5.5Anthropic Claude Sonnet 5.5search off | 100%45 of 45 discovery answers | No linksNo cited links | 11.6%33 of 285 checkable claims | 0.9341 of 50 questions alike in all 3 runs |
| Gemini 3.8 FlashGoogle Gemini 3.8 Flashgrounding on | 100%45 of 45 discovery answers | 57.5%506 of 880 checked links | 11.9%38 of 318 checkable claims | 0.9139 of 50 questions alike in all 3 runs |
| Gemini 3.8 FlashGoogle Gemini 3.8 Flashgrounding off | 100%45 of 45 discovery answers | 0%0 of 1 checked link | 18.1%55 of 304 checkable claims | 0.9240 of 50 questions alike in all 3 runs |
| SonarPerplexity Sonar (via OpenRouter)search always on | 100%45 of 45 discovery answers | 37.1%1,045 of 2,820 checked links | 7.6%15 of 197 checkable claims | 0.8735 of 50 questions alike in all 3 runs |
- Brand mentions
- Discovery answers that name at least one panel brand from the question's category.
- Citation accuracy
- Checked cited links that resolve to a live page and support the sentence they are attached to.
- Factual error rate
- Checkable claims that contradict the dated fact sheet of the brand's own pages.
- Consistency
- The most common outcome's share of a question's runs, averaged over the questions (0 to 1).
02By category
Category by configuration
The same four measures split by product category. Pick the measure to emphasise; every cell still carries all four values.
0%26.2%: colour runs from the lowest to the highest factual error rate in this view
Factual error rate: Checkable claims that contradict the dated fact sheet of the brand's own pages. Colour shows the value within this view, from the lowest to the highest; it is not a score.
| Configuration | CRM software | Payroll software | Project management software | Online travel booking | Ecommerce platforms |
|---|---|---|---|---|---|
| GPT-6 Solsearch on | Factual error rate: 17.6%Mentions: 100%Citation accuracy: 45.9%Consistency: 1.00 | Factual error rate: 0%Mentions: 100%Citation accuracy: 47.4%Consistency: 0.90 | Factual error rate: 0%Mentions: 100%Citation accuracy: 35.3%Consistency: 0.97 | Factual error rate: 0%Mentions: 88.9%Citation accuracy: 55.8%Consistency: 0.93 | Factual error rate: 0%Mentions: 88.9%Citation accuracy: 39%Consistency: 0.90 |
| GPT-6 Solsearch off | Factual error rate: 12.9%Mentions: 100%Citation accuracy: No linksConsistency: 1.00 | Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.90 | Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.97 | Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.93 | Factual error rate: 16.7%Mentions: 100%Citation accuracy: No linksConsistency: 0.90 |
| Claude Sonnet 5.5search on | Factual error rate: 20%Mentions: 100%Citation accuracy: 91.1%Consistency: 0.93 | Factual error rate: 0%Mentions: 100%Citation accuracy: 92.5%Consistency: 0.93 | Factual error rate: 4.1%Mentions: 100%Citation accuracy: 90.8%Consistency: 0.97 | Factual error rate: 0%Mentions: 100%Citation accuracy: 93.6%Consistency: 0.93 | Factual error rate: 14.5%Mentions: 100%Citation accuracy: 90.5%Consistency: 1.00 |
| Claude Sonnet 5.5search off | Factual error rate: 23.9%Mentions: 100%Citation accuracy: No linksConsistency: 0.93 | Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.87 | Factual error rate: 11.1%Mentions: 100%Citation accuracy: No linksConsistency: 0.97 | Factual error rate: 0%Mentions: 100%Citation accuracy: No linksConsistency: 0.93 | Factual error rate: 21.7%Mentions: 100%Citation accuracy: No linksConsistency: 0.97 |
| Gemini 3.8 Flashgrounding on | Factual error rate: 16.4%Mentions: 100%Citation accuracy: 55.6%Consistency: 0.90 | Factual error rate: 0%Mentions: 100%Citation accuracy: 64.9%Consistency: 0.90 | Factual error rate: 13.2%Mentions: 100%Citation accuracy: 69.6%Consistency: 0.97 | Factual error rate: 10%Mentions: 100%Citation accuracy: 75.6%Consistency: 0.90 | Factual error rate: 18.6%Mentions: 100%Citation accuracy: 49.3%Consistency: 0.90 |
| Gemini 3.8 Flashgrounding off | Factual error rate: 26.2%Mentions: 100%Citation accuracy: No linksConsistency: 0.87 | Factual error rate: 0%Mentions: 100%Citation accuracy: 0%Consistency: 0.93 | Factual error rate: 16.7%Mentions: 100%Citation accuracy: No linksConsistency: 0.97 | Factual error rate: 14.8%Mentions: 100%Citation accuracy: No linksConsistency: 0.90 | Factual error rate: 25.7%Mentions: 100%Citation accuracy: No linksConsistency: 0.93 |
| Sonarsearch always on | Factual error rate: 15%Mentions: 100%Citation accuracy: 43%Consistency: 0.87 | Factual error rate: 0%Mentions: 100%Citation accuracy: 35.8%Consistency: 0.87 | Factual error rate: 2.1%Mentions: 100%Citation accuracy: 36.7%Consistency: 0.97 | Factual error rate: 0%Mentions: 100%Citation accuracy: 30.8%Consistency: 0.87 | Factual error rate: 18.6%Mentions: 100%Citation accuracy: 37.5%Consistency: 0.80 |
Columns: CRM = CRM software · Payroll = Payroll software · Project = Project management software · Travel = Online travel booking · Ecommerce = Ecommerce platforms. The other three measures for each cell are in the full table under this chart and in the data download.
Show every category as a table
| Configuration | Brand mentions | Citation accuracy | Factual error rate | Consistency |
|---|---|---|---|---|
| OpenAI GPT-6 Sol — web search on | 100%9 of 9 discovery answers | 45.9%28 of 61 checked links | 17.6%9 of 51 checkable claims | 1.0010 of 10 questions alike in all 3 runs |
| OpenAI GPT-6 Sol — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 12.9%4 of 31 checkable claims | 1.0010 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search on | 100%9 of 9 discovery answers | 91.1%288 of 316 checked links | 20%11 of 55 checkable claims | 0.939 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 23.9%11 of 46 checkable claims | 0.938 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — Google Search grounding on | 100%9 of 9 discovery answers | 55.6%183 of 329 checked links | 16.4%10 of 61 checkable claims | 0.907 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — grounding off | 100%9 of 9 discovery answers | No linksNo cited links | 26.2%17 of 65 checkable claims | 0.877 of 10 questions alike in all 3 runs |
| Perplexity Sonar (via OpenRouter) — search always on | 100%9 of 9 discovery answers | 43%278 of 647 checked links | 15%6 of 40 checkable claims | 0.877 of 10 questions alike in all 3 runs |
| Configuration | Brand mentions | Citation accuracy | Factual error rate | Consistency |
|---|---|---|---|---|
| OpenAI GPT-6 Sol — web search on | 100%9 of 9 discovery answers | 47.4%18 of 38 checked links | 0%0 of 42 checkable claims | 0.908 of 10 questions alike in all 3 runs |
| OpenAI GPT-6 Sol — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 0%0 of 30 checkable claims | 0.907 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search on | 100%9 of 9 discovery answers | 92.5%185 of 200 checked links | 0%0 of 38 checkable claims | 0.938 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 0%0 of 46 checkable claims | 0.877 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — Google Search grounding on | 100%9 of 9 discovery answers | 64.9%48 of 74 checked links | 0%0 of 61 checkable claims | 0.907 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — grounding off | 100%9 of 9 discovery answers | 0%0 of 1 checked link | 0%0 of 43 checkable claims | 0.939 of 10 questions alike in all 3 runs |
| Perplexity Sonar (via OpenRouter) — search always on | 100%9 of 9 discovery answers | 35.8%120 of 335 checked links | 0%0 of 30 checkable claims | 0.877 of 10 questions alike in all 3 runs |
| Configuration | Brand mentions | Citation accuracy | Factual error rate | Consistency |
|---|---|---|---|---|
| OpenAI GPT-6 Sol — web search on | 100%9 of 9 discovery answers | 35.3%24 of 68 checked links | 0%0 of 81 checkable claims | 0.979 of 10 questions alike in all 3 runs |
| OpenAI GPT-6 Sol — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 0%0 of 42 checkable claims | 0.979 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search on | 100%9 of 9 discovery answers | 90.8%277 of 305 checked links | 4.1%3 of 73 checkable claims | 0.979 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 11.1%9 of 81 checkable claims | 0.979 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — Google Search grounding on | 100%9 of 9 discovery answers | 69.6%96 of 138 checked links | 13.2%10 of 76 checkable claims | 0.979 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — grounding off | 100%9 of 9 discovery answers | No linksNo cited links | 16.7%12 of 72 checkable claims | 0.979 of 10 questions alike in all 3 runs |
| Perplexity Sonar (via OpenRouter) — search always on | 100%9 of 9 discovery answers | 36.7%229 of 624 checked links | 2.1%1 of 48 checkable claims | 0.979 of 10 questions alike in all 3 runs |
| Configuration | Brand mentions | Citation accuracy | Factual error rate | Consistency |
|---|---|---|---|---|
| OpenAI GPT-6 Sol — web search on | 88.9%8 of 9 discovery answers | 55.8%24 of 43 checked links | 0%0 of 30 checkable claims | 0.938 of 10 questions alike in all 3 runs |
| OpenAI GPT-6 Sol — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 0%0 of 30 checkable claims | 0.938 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search on | 100%9 of 9 discovery answers | 93.6%146 of 156 checked links | 0%0 of 47 checkable claims | 0.939 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 0%0 of 52 checkable claims | 0.938 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — Google Search grounding on | 100%9 of 9 discovery answers | 75.6%34 of 45 checked links | 10%5 of 50 checkable claims | 0.908 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — grounding off | 100%9 of 9 discovery answers | No linksNo cited links | 14.8%8 of 54 checkable claims | 0.907 of 10 questions alike in all 3 runs |
| Perplexity Sonar (via OpenRouter) — search always on | 100%9 of 9 discovery answers | 30.8%173 of 561 checked links | 0%0 of 36 checkable claims | 0.877 of 10 questions alike in all 3 runs |
| Configuration | Brand mentions | Citation accuracy | Factual error rate | Consistency |
|---|---|---|---|---|
| OpenAI GPT-6 Sol — web search on | 88.9%8 of 9 discovery answers | 39%23 of 59 checked links | 0%0 of 69 checkable claims | 0.908 of 10 questions alike in all 3 runs |
| OpenAI GPT-6 Sol — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 16.7%6 of 36 checkable claims | 0.907 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search on | 100%9 of 9 discovery answers | 90.5%256 of 283 checked links | 14.5%9 of 62 checkable claims | 1.0010 of 10 questions alike in all 3 runs |
| Anthropic Claude Sonnet 5.5 — web search off | 100%9 of 9 discovery answers | No linksNo cited links | 21.7%13 of 60 checkable claims | 0.979 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — Google Search grounding on | 100%9 of 9 discovery answers | 49.3%145 of 294 checked links | 18.6%13 of 70 checkable claims | 0.908 of 10 questions alike in all 3 runs |
| Google Gemini 3.8 Flash — grounding off | 100%9 of 9 discovery answers | No linksNo cited links | 25.7%18 of 70 checkable claims | 0.938 of 10 questions alike in all 3 runs |
| Perplexity Sonar (via OpenRouter) — search always on | 100%9 of 9 discovery answers | 37.5%245 of 653 checked links | 18.6%8 of 43 checkable claims | 0.805 of 10 questions alike in all 3 runs |
03Search on vs off
The same models with search on and off
Three models were run with search on and off. Each row joins a model's search-on value to its search-off value for the same questions.
Factual error rate
Checkable claims that contradict the dated fact sheet. A dot further left means fewer contradicted claims.
Citation accuracy
Checked cited links that resolve to a live page and support their sentence. A configuration whose answers cited no links shows “No links” and has no dot.
Show the data as a table
| Model | Factual errors, on | Factual errors, off | Citation accuracy, on | Citation accuracy, off |
|---|---|---|---|---|
| OpenAI GPT-6 Sol | 3.3%9 of 273 checkable claims | 5.9%10 of 169 checkable claims | 43.5%117 of 269 checked links | No linksNo cited links |
| Anthropic Claude Sonnet 5.5 | 8.4%23 of 275 checkable claims | 11.6%33 of 285 checkable claims | 91.4%1,152 of 1,260 checked links | No linksNo cited links |
| Google Gemini 3.8 Flash | 11.9%38 of 318 checkable claims | 18.1%55 of 304 checkable claims | 57.5%506 of 880 checked links | 0%0 of 1 checked link |
04Consistency
Repeated runs, lap by lap
Every question was sent 3 times to each configuration. Each lap is one question and each dot one run, grouped by category.
- GPT-6 Solweb search on · mean consistency 0.94All 3 runs alike: 43 of 50 questions
- GPT-6 Solweb search off · mean consistency 0.94All 3 runs alike: 41 of 50 questions
- Claude Sonnet 5.5web search on · mean consistency 0.95All 3 runs alike: 45 of 50 questions
- Claude Sonnet 5.5web search off · mean consistency 0.93All 3 runs alike: 41 of 50 questions
- Gemini 3.8 FlashGoogle Search grounding on · mean consistency 0.91All 3 runs alike: 39 of 50 questions
- Gemini 3.8 Flashgrounding off · mean consistency 0.92All 3 runs alike: 40 of 50 questions
- Sonarsearch always on · mean consistency 0.87All 3 runs alike: 35 of 50 questions
Show the data as a table
| Configuration | Mean consistency | All 3 runs alike | 2 of 3 runs alike | No two runs alike |
|---|---|---|---|---|
| OpenAI GPT-6 Sol — web search on | 0.94 | 43 of 50 questions | 5 of 50 questions | 2 of 50 questions |
| OpenAI GPT-6 Sol — web search off | 0.94 | 41 of 50 questions | 9 of 50 questions | 0 of 50 questions |
| Anthropic Claude Sonnet 5.5 — web search on | 0.95 | 45 of 50 questions | 3 of 50 questions | 2 of 50 questions |
| Anthropic Claude Sonnet 5.5 — web search off | 0.93 | 41 of 50 questions | 8 of 50 questions | 1 of 50 questions |
| Google Gemini 3.8 Flash — Google Search grounding on | 0.91 | 39 of 50 questions | 9 of 50 questions | 2 of 50 questions |
| Google Gemini 3.8 Flash — grounding off | 0.92 | 40 of 50 questions | 8 of 50 questions | 2 of 50 questions |
| Perplexity Sonar (via OpenRouter) — search always on | 0.87 | 35 of 50 questions | 11 of 50 questions | 4 of 50 questions |
05Brand naming
Which panel brands the answers named
Discovery questions only; usable answers in the brand's own category. Counts are what the answers showed; they do not explain why a brand was or was not named.
CRM software
- SalesforceNamed in 39 of 63 answers
- Microsoft Dynamics 365 SalesNamed in 5 of 63 answers
- Zoho CRMNamed in 49 of 63 answers
- PipedriveNamed in 43 of 63 answers
- FreshsalesNamed in 24 of 63 answers
Counts per configuration
| Brand | GPT-6 Sol, search on | GPT-6 Sol, search off | Claude Sonnet 5.5, search on | Claude Sonnet 5.5, search off | Gemini 3.8 Flash, grounding on | Gemini 3.8 Flash, grounding off | Sonar, search always on |
|---|---|---|---|---|---|---|---|
| Salesforce | 3 of 9 | 3 of 9 | 9 of 9 | 6 of 9 | 5 of 9 | 4 of 9 | 9 of 9 |
| Microsoft Dynamics 365 Sales | 0 of 9 | 0 of 9 | 2 of 9 | 3 of 9 | 0 of 9 | 0 of 9 | 0 of 9 |
| Zoho CRM | 9 of 9 | 6 of 9 | 8 of 9 | 8 of 9 | 6 of 9 | 6 of 9 | 6 of 9 |
| Pipedrive | 6 of 9 | 6 of 9 | 6 of 9 | 8 of 9 | 5 of 9 | 6 of 9 | 6 of 9 |
| Freshsales | 3 of 9 | 3 of 9 | 3 of 9 | 6 of 9 | 3 of 9 | 4 of 9 | 2 of 9 |
Payroll software
- GustoNamed in 62 of 63 answers
- ADPNamed in 46 of 63 answers
- PaychexNamed in 17 of 63 answers
- RipplingNamed in 27 of 63 answers
- QuickBooks PayrollNamed in 34 of 63 answers
Counts per configuration
| Brand | GPT-6 Sol, search on | GPT-6 Sol, search off | Claude Sonnet 5.5, search on | Claude Sonnet 5.5, search off | Gemini 3.8 Flash, grounding on | Gemini 3.8 Flash, grounding off | Sonar, search always on |
|---|---|---|---|---|---|---|---|
| Gusto | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 8 of 9 |
| ADP | 3 of 9 | 5 of 9 | 9 of 9 | 9 of 9 | 8 of 9 | 5 of 9 | 7 of 9 |
| Paychex | 0 of 9 | 0 of 9 | 8 of 9 | 6 of 9 | 0 of 9 | 0 of 9 | 3 of 9 |
| Rippling | 2 of 9 | 2 of 9 | 3 of 9 | 4 of 9 | 6 of 9 | 6 of 9 | 4 of 9 |
| QuickBooks Payroll | 3 of 9 | 6 of 9 | 6 of 9 | 7 of 9 | 4 of 9 | 5 of 9 | 3 of 9 |
Project management software
- AsanaNamed in 63 of 63 answers
- monday.comNamed in 35 of 63 answers
- TrelloNamed in 51 of 63 answers
- ClickUpNamed in 62 of 63 answers
- SmartsheetNamed in 2 of 63 answers
Counts per configuration
| Brand | GPT-6 Sol, search on | GPT-6 Sol, search off | Claude Sonnet 5.5, search on | Claude Sonnet 5.5, search off | Gemini 3.8 Flash, grounding on | Gemini 3.8 Flash, grounding off | Sonar, search always on |
|---|---|---|---|---|---|---|---|
| Asana | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 |
| monday.com | 0 of 9 | 3 of 9 | 7 of 9 | 9 of 9 | 5 of 9 | 5 of 9 | 6 of 9 |
| Trello | 6 of 9 | 6 of 9 | 9 of 9 | 9 of 9 | 6 of 9 | 6 of 9 | 9 of 9 |
| ClickUp | 8 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 |
| Smartsheet | 0 of 9 | 0 of 9 | 0 of 9 | 0 of 9 | 0 of 9 | 0 of 9 | 2 of 9 |
Online travel booking
- Booking.comNamed in 39 of 63 answers
- ExpediaNamed in 38 of 63 answers
- PricelineNamed in 27 of 63 answers
- Trip.comNamed in 22 of 63 answers
- AgodaNamed in 35 of 63 answers
Counts per configuration
| Brand | GPT-6 Sol, search on | GPT-6 Sol, search off | Claude Sonnet 5.5, search on | Claude Sonnet 5.5, search off | Gemini 3.8 Flash, grounding on | Gemini 3.8 Flash, grounding off | Sonar, search always on |
|---|---|---|---|---|---|---|---|
| Booking.com | 3 of 9 | 4 of 9 | 7 of 9 | 7 of 9 | 5 of 9 | 6 of 9 | 7 of 9 |
| Expedia | 3 of 9 | 3 of 9 | 6 of 9 | 6 of 9 | 7 of 9 | 8 of 9 | 5 of 9 |
| Priceline | 3 of 9 | 2 of 9 | 5 of 9 | 4 of 9 | 5 of 9 | 4 of 9 | 4 of 9 |
| Trip.com | 3 of 9 | 3 of 9 | 3 of 9 | 3 of 9 | 3 of 9 | 3 of 9 | 4 of 9 |
| Agoda | 3 of 9 | 3 of 9 | 6 of 9 | 5 of 9 | 6 of 9 | 6 of 9 | 6 of 9 |
Ecommerce platforms
- ShopifyNamed in 55 of 63 answers
- WooCommerceNamed in 56 of 63 answers
- BigCommerceNamed in 33 of 63 answers
- EcwidNamed in 18 of 63 answers
- Square OnlineNamed in 33 of 63 answers
Counts per configuration
| Brand | GPT-6 Sol, search on | GPT-6 Sol, search off | Claude Sonnet 5.5, search on | Claude Sonnet 5.5, search off | Gemini 3.8 Flash, grounding on | Gemini 3.8 Flash, grounding off | Sonar, search always on |
|---|---|---|---|---|---|---|---|
| Shopify | 7 of 9 | 7 of 9 | 9 of 9 | 9 of 9 | 8 of 9 | 8 of 9 | 7 of 9 |
| WooCommerce | 6 of 9 | 6 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 9 of 9 | 8 of 9 |
| BigCommerce | 3 of 9 | 4 of 9 | 6 of 9 | 6 of 9 | 4 of 9 | 4 of 9 | 6 of 9 |
| Ecwid | 0 of 9 | 0 of 9 | 6 of 9 | 3 of 9 | 2 of 9 | 3 of 9 | 4 of 9 |
| Square Online | 1 of 9 | 5 of 9 | 6 of 9 | 6 of 9 | 5 of 9 | 5 of 9 | 5 of 9 |
06Annotated examples
What the checks look like on real answers
Excerpts from collected answers, each beside the fact-sheet value and the brand's own page it was checked against.
Anthropic Claude Sonnet 5.5 — web search on
Question Does Zoho CRM have a free plan, and how many users can use it?
“Zoho CRM has a free edition that supports up to 3 users.”
Cited link https://www.zoho.com/en-us/crm/zohocrm-pricing.htmlLive page (HTTP 200) that supports the sentence
- Fact sheet · Zoho CRM · Free plan
- Yes — free edition for up to 3 users
- First-party source · fact sheet dated October 23, 2026
- https://www.zoho.com/crm/zohocrm-pricing.html
- Grading
- Matches the fact sheet (both graders agreed)
OpenAI GPT-6 Sol — web search on
Question Does Zoho CRM have a free plan, and how many users can use it?
“Zoho CRM has a free plan for up to 3 users.”
Cited link https://www.zoho.com/sites/default/files/bigin/bigin-zohocrm-editions-comparison.pdf?utm_source=openaiDead link: HTTP 404 when checked in the run week
- Fact sheet · Zoho CRM · Free plan
- Yes — free edition for up to 3 users
- First-party source · fact sheet dated October 23, 2026
- https://www.zoho.com/crm/zohocrm-pricing.html
- Grading
- Matches the fact sheet (both graders agreed)
OpenAI GPT-6 Sol — web search off
Question What plans does BigCommerce offer, and does it have a free plan?
“BigCommerce offers four plans: Standard, Plus, Pro, and Enterprise”
- Fact sheet · BigCommerce · Plan names
- Core, Growth, Scale, Performance
- First-party source · fact sheet dated October 23, 2026
- https://www.bigcommerce.com/essentials/pricing/
- Grading
- Contradicts the fact sheet (both graders agreed)
07Methodology
Method and run conditions
Scope. These are answers from each provider's API, not consumer-app sessions, and they are not live Google AI Overviews or Google AI Mode. Google appears here only as Gemini, with Google Search grounding switched on or off.
Run conditions
| Configuration | Provider and API | Requested model | Returned version | Search | Answers collected | Runs |
|---|---|---|---|---|---|---|
| OpenAI GPT-6 Solsearch on | OpenAI · Responses API | gpt-6-sol | gpt-6-sol | OpenAI web_search toolSearched in 150 of 150 returned answers | October 23, 2026, 11:22 to 11:53 UTC | 3 |
| OpenAI GPT-6 Solsearch off | OpenAI · Responses API | gpt-6-sol | gpt-6-sol | None | October 23, 2026, 11:22 to 11:57 UTC | 3 |
| Anthropic Claude Sonnet 5.5search on | Anthropic · Messages API | claude-sonnet-5-5 | claude-sonnet-5-5 | Anthropic web_search tool (web_search_20250305)Searched in 130 of 150 returned answers | October 23, 2026, 11:22 to 11:52 UTC | 3 |
| Anthropic Claude Sonnet 5.5search off | Anthropic · Messages API | claude-sonnet-5-5 | claude-sonnet-5-5 | None | October 23, 2026, 11:52 to 11:58 UTC | 3 |
| Google Gemini 3.8 Flashgrounding on | Google · Gemini API generateContent | gemini-3.8-flash | gemini-3.8-flash | Grounding with Google Search (googleSearch tool)Searched in 64 of 150 returned answers | October 23, 2026, 11:23 to 12:01 UTC | 3 |
| Google Gemini 3.8 Flashgrounding off | Google · Gemini API generateContent | gemini-3.8-flash | gemini-3.8-flash | None | October 23, 2026, 12:01 to 12:21 UTC | 3 |
| Perplexity Sonar (via OpenRouter)search always on | Perplexity · OpenRouter Chat Completions | perplexity/sonar | perplexity/sonar | Sonar's built-in web search (cannot be switched off) | October 23, 2026, 11:23 to 11:40 UTC | 3 |
Prompting. Each question is sent exactly as written in the question set, as a single user message. No system prompt, no instructions, no conversation history. Nothing else is added.
Market. Questions are written in US English. Where an API accepts a location hint for its search tool, the hint is country US; otherwise no location is sent.
Sampling settings per configuration
| Configuration | Sampling |
|---|---|
| OpenAI GPT-6 Sol — web search on | Provider defaults: temperature, top_p and reasoning effort are not sent. |
| OpenAI GPT-6 Sol — web search off | Provider defaults: temperature, top_p and reasoning effort are not sent. |
| Anthropic Claude Sonnet 5.5 — web search on | Provider defaults: temperature and top_p are not sent; extended thinking is not enabled. |
| Anthropic Claude Sonnet 5.5 — web search off | Provider defaults: temperature and top_p are not sent; extended thinking is not enabled. |
| Google Gemini 3.8 Flash — Google Search grounding on | Provider defaults: temperature, topP and thinking settings are not sent. The googleSearch tool takes no location parameter, so no location hint is sent. |
| Google Gemini 3.8 Flash — grounding off | Provider defaults: temperature, topP and thinking settings are not sent. |
| Perplexity Sonar (via OpenRouter) — search always on | Provider defaults: temperature, top_p and search options are not sent. Sonar has no search-off mode, so this configuration has no search-off pair. |
Answers requested, returned, usable and missing
- Requested
- 1,050
- Returned
- 1,050
- Usable
- 1,050
- Missing
- 0
Answer counts per configuration
| Configuration | Requested | Returned | Usable | Missing |
|---|---|---|---|---|
| OpenAI GPT-6 Sol — web search on | 150 | 150 | 150 | 0 |
| OpenAI GPT-6 Sol — web search off | 150 | 150 | 150 | 0 |
| Anthropic Claude Sonnet 5.5 — web search on | 150 | 150 | 150 | 0 |
| Anthropic Claude Sonnet 5.5 — web search off | 150 | 150 | 150 | 0 |
| Google Gemini 3.8 Flash — Google Search grounding on | 150 | 150 | 150 | 0 |
| Google Gemini 3.8 Flash — grounding off | 150 | 150 | 150 | 0 |
| Perplexity Sonar (via OpenRouter) — search always on | 150 | 150 | 150 | 0 |
Definitions
- Brand named
- Answers that explicitly name the brand or an approved name variant.
- Consistency
- Consistency = most-common outcome count ÷ number of runs
- Fact accuracy
- What the answer says about you matches what your page says: name, price, dates, location, product limits.
- Source accuracy
- The link under the answer goes to the right page on your site, not an old URL, a copy or a third-party page.
- Citation accuracy
- Benchmark measure: the share of cited links that resolve to a live page and support the claim they are attached to. Dead links, links that redirect to a homepage and unsupported links are counted separately; links that could not be checked are reported separately and left out of the denominator. A link attached to several sentences is counted once for each sentence. It checks every cited link on any site, so it is not the same as the Journal's split of "citation accuracy" into source, fact and attribution accuracy.
- Citation accuracy = supported links ÷ (supported + unsupported + dead + redirected-to-homepage links)
- Factual error rate
- Benchmark measure: checkable claims that contradict the dated fact sheet ÷ checkable claims. A claim is checkable when the fact sheet covers it. Two grader models grade every claim; disagreements are checked against the brand's first-party source. Unresolved claims are left out of the rate and counted.
- Factual error rate = claims that contradict the fact sheet ÷ checkable claims (unresolved claims excluded and counted)
- Requested / returned / usable / missing
- Requested: answers the edition asked for (questions × runs × configurations). Returned: the API returned an answer with text. Usable: a returned answer that responds to the question (refusals, clarification requests and off-topic replies are returned but not usable). Missing: the run produced no answer text (failed call, timeout or empty response). Percentages use the stated denominator.
Question types and what counts as the same outcome
- Discovery15 questions
- The set of this category's panel brands the answer names (approved variants count; order and wording ignored). Two runs share an outcome when they name exactly the same set of panel brands, including the empty set.
- Brand fact25 questions
- For each fact field the question asks about, the answer's resolved verdict against the fact sheet: consistent, contradicts, not stated or unresolved. Two runs share an outcome when every asked field has the same verdict.
- Head to head10 questions
- The pick the answer states: the first brand, the second brand, or no single pick (the answer declines to choose or says it depends). Two runs share an outcome when they state the same pick.
How scoring and grading work
| Role | Model | What it does |
|---|---|---|
| Extractor | gemini-3.8-flash | Classifies usability, extracts checkable fact claims and the head-to-head pick from each answer. |
| Grader A | gpt-6-luna | Grader A for every checkable claim against the fact sheet. |
| Grader B | claude-haiku-5-5 | Grader B for every checkable claim against the fact sheet. |
| Support judge | gemini-3.8-flash | Judges whether a live cited page supports the answer text the link is attached to. |
- 1,914 fact claims extracted; 1,821 were checkable against the fact sheet.
- 82 grader disagreements, 82 resolved against the brand's first-party source snapshot, 0 unresolved.
- 38 verdicts both graders agreed on were changed in the sanity check (37 to not checkable, 1 to consistent).
Disagreements between the two graders were checked against the brand's first-party source snapshot and recorded in a private review sheet. The sanity check also re-read every claim both graders marked as an error; verdicts it changed are counted here and logged in the same sheet.
Scoring calls are batched because the integrations proxy rate-limits: the extractor reads the runs of one (configuration, question) together, graders take up to 25 claims per call, the support judge up to 6 (page, claim) pairs per call.
Scoring ran October 23, 2026, 11:37 to 13:57 UTC.
The fact sheet
Dated October 23, 2026. 55 facts about 25 brands, from 38 snapshots of the brands' own pages taken October 23, 2026, 11:06 to 11:13 UTC.
A claim is an error only when it contradicts the fact as stated here. A claim that is incomplete but compatible, or about something this sheet does not cover, is not an error (it is either consistent or not checkable).
Show every fact and its source
| Brand | Field | Fact | Source |
|---|---|---|---|
| Salesforce | Founded | 1999 | https://www.salesforce.com/company/about-us/ |
| Salesforce | Free plan | Yes — Free Suite, $0, comes with 2 user licenses | https://www.salesforce.com/crm/free-crm/ |
| Salesforce | Headquarters | San Francisco, California | https://www.salesforce.com/company/locations/ |
| Microsoft Dynamics 365 Sales | Free plan | No — free trial only | https://www.microsoft.com/en-us/dynamics-365/products/sales/pricing |
| Microsoft Dynamics 365 Sales | Plan names | Sales Professional, Sales Enterprise, Sales Premium, Microsoft Relationship Sales | https://www.microsoft.com/en-us/dynamics-365/products/sales/pricing |
| Zoho CRM | Free plan | Yes — free edition for up to 3 users | https://www.zoho.com/crm/zohocrm-pricing.html |
| Pipedrive | Founded | 2010 | https://www.pipedrive.com/en/about |
| Pipedrive | Headquarters | New York, NY | https://www.pipedrive.com/en/about |
| Pipedrive | Free plan | No — 14-day free trial; paid plans Lite, Growth, Premium, Ultimate | https://www.pipedrive.com/en/pricing |
| Freshsales | Free plan | No — 21-day free trial, then a paid plan | https://www.freshworks.com/crm/pricing/ |
| Freshsales | Plan names | Growth, Pro, Enterprise | https://www.freshworks.com/crm/pricing/ |
| Freshsales | Headquarters | San Mateo, California (Freshworks) | https://ir.freshworks.com/resources/investor-faqs/default.aspx |
| Brand | Field | Fact | Source |
|---|---|---|---|
| Gusto | Plan names | Simple, Plus, Premium (plus a separate plan for paying yourself) | https://gusto.com/product/pricing |
| Gusto | Former name | ZenPayroll (launched in 2012) | https://gusto.com/about |
| ADP | Headquarters | Roseland, New Jersey | https://mediacenter.adp.com/image/ADP+Corporate+Fact+Sheet.pdf |
| ADP | Founded | 1949 | https://mediacenter.adp.com/image/ADP+Corporate+Fact+Sheet.pdf |
| Paychex | Headquarters | Dual headquarters: Rochester, NY and Cincinnati, OH | https://www.paychex.com/corporate |
| Paychex | Founded | 1971 | https://www.paychex.com/corporate |
| Rippling | Headquarters | San Francisco, California | https://www.rippling.com/press |
| Rippling | CEO | Parker Conrad (co-founder and CEO) | https://www.rippling.com/about |
| QuickBooks Payroll | Parent company | Intuit | https://www.intuit.com/company/ |
| QuickBooks Payroll | Related products | TurboTax, Credit Karma, Mailchimp, Intuit Enterprise Suite | https://www.intuit.com/company/ |
| Brand | Field | Fact | Source |
|---|---|---|---|
| Asana | Free plan | Yes — Personal plan, $0, for one or two people | https://asana.com/pricing |
| Asana | Headquarters | San Francisco, California | https://asana.com/company |
| Asana | Plan names | Personal, Starter, Advanced, Enterprise (Enterprise+ for advanced security) | https://asana.com/pricing |
| monday.com | Free plan | Yes — Free plan, $0, up to 2 seats | https://monday.com/pricing |
| monday.com | Plan names | Free, Basic, Standard, Pro, Enterprise | https://monday.com/pricing |
| Trello | Free plan | Yes — Free, $0, up to 10 collaborators per Workspace | https://trello.com/pricing |
| Trello | Parent company | Atlassian (acquired early 2017) | https://trello.com/about |
| Trello | Launch year | 2011 | https://trello.com/about |
| ClickUp | Headquarters | San Diego, California | https://clickup.com/about |
| ClickUp | Free plan | Yes — Free Forever plan | https://clickup.com/pricing |
| Smartsheet | Founded | 2005 | https://www.smartsheet.com/about |
| Smartsheet | Headquarters | Bellevue, Washington | https://www.smartsheet.com/about |
| Brand | Field | Fact | Source |
|---|---|---|---|
| Booking.com | Headquarters | Amsterdam, the Netherlands | https://www.booking.com/content/about.html |
| Booking.com | Parent company | Booking Holdings Inc. | https://www.booking.com/content/about.html |
| Booking.com | Founded | 1996 | https://www.booking.com/content/about.html |
| Expedia | Founded | 1996 | https://www.expediagroup.com/en-us/about-us |
| Expedia | Origin | Started inside Microsoft | https://www.expediagroup.com/en-us/about-us |
| Priceline | Launch year | 1998 | https://press.priceline.com/about-priceline |
| Priceline | Parent company | Booking Holdings | https://www.bookingholdings.com/about/ |
| Trip.com | Founded | 1999 | https://group.trip.com/about/story?locale=en-US |
| Trip.com | Former name | Ctrip (www.ctrip.com) | https://group.trip.com/about/story?locale=en-US |
| Agoda | Headquarters | Singapore | https://www.agoda.com/info/about-agoda.html |
| Agoda | Parent company | Booking Holdings | https://www.agoda.com/info/about-agoda.html |
| Brand | Field | Fact | Source |
|---|---|---|---|
| Shopify | Plan names | Basic, Grow, Advanced, Plus | https://www.shopify.com/pricing |
| Shopify | Headquarters | Las Vegas, Nevada (principal executive office) | https://shopify.com/investors/investor-faqs |
| WooCommerce | Free plan | Yes — the core platform is free and open source | https://woocommerce.com/about/ |
| WooCommerce | Parent company | Woo, part of Automattic (acquired 2015) | https://woocommerce.com/about/ |
| BigCommerce | Plan names | Core, Growth, Scale, Performance | https://www.bigcommerce.com/essentials/pricing/ |
| BigCommerce | Free plan | No — free trial only | https://www.bigcommerce.com/essentials/pricing/ |
| Ecwid | Free plan | No — plans are Starter, Venture, Business and Unlimited, all paid | https://www.ecwid.com/pricing |
| Ecwid | Parent company | Lightspeed | https://support.ecwid.com/hc/en-us/articles/5582039267996-Ecwid-is-now-Lightspeed |
| Square Online | Free plan | Yes — Square Free, $0/month per location | https://squareup.com/us/en/online-store/plans |
| Square Online | Plan names | Square Free, Square Plus, Square Premium | https://squareup.com/us/en/online-store/plans |
Limitations
- API answers only. These are not captures of the ChatGPT, Claude, Gemini or Perplexity apps, of Google AI Overviews or of Google AI Mode, and consumer apps can answer differently.
- One market and language (United States, US English) and one collection week. Answers change over time and can differ by location.
- Three runs per question per configuration, so a question's consistency can only be 1/3, 2/3 or 1. It shows variation in this sample, not a long-run rate.
- Each question was sent as written with provider default settings and no system prompt. Other settings, prompts or follow-up questions can produce different answers.
- The factual error rate covers only facts in the dated fact sheet, taken from each brand's own pages in the run week. A claim counts as an error when it contradicts the brand's own page as snapshotted; statements on topics outside the fact sheet were not graded.
- Each fact is checked against the page named as its source. Where a brand describes itself differently elsewhere (for example a company that names one principal executive office but is widely described as based in another city), answers that repeat the other description count as contradicting the named page. Plans or offices that the source page does not list were treated as not checkable rather than as errors.
- Claims, picks and citation support were read by AI models (an extractor, two graders from different providers and a support judge). Grader disagreements were checked against the first-party snapshot, and every claim both graders marked as an error was re-read in a sanity check; errors that both graders missed in the same way can remain.
- Cited links were checked with a plain HTTP request (no browser) in the run week. Pages that blocked automated requests, returned too little readable text without a browser, or were PDFs are counted as unverifiable and left out of the citation-accuracy denominator. Pages that load part of their content with scripts were judged on the text the plain request returned, so a few supporting passages may have been missed.
- With Google Search grounding on, Gemini decides whether to search; some grounding-on answers ran no search at all and are still counted in that configuration.
- Perplexity Sonar was reached through OpenRouter rather than Perplexity's own API; Sonar always searches, so it has no search-off pair.
- The panel is 25 well-known brands in five categories. Leading brands that sell AI-visibility products (for example HubSpot in CRM) were left out, so answers that name them are not scored for them. Results do not describe smaller or lesser-known brands.
Not included
- Perplexity Sonar with a direct Perplexity API key
- Not needed: Sonar was reachable through the Replit-managed OpenRouter route, so no key was requested. OpenRouter forwards to Perplexity; answers may differ slightly from Perplexity's own API.
- Consumer apps (ChatGPT app, Claude app, Gemini app, Google AI Overviews, Google AI Mode, Perplexity app)
- Out of scope. The benchmark measures API answers only; they are not captures of any consumer app or search results page.
- Search-off for Perplexity Sonar
- Sonar always searches; the API offers no search-off mode.
Comparability. First edition: there is no previous edition to compare against.
08Question set
The full question set
Show all 50 questionsQuestion set v1, grouped by category, exactly as sent
CRM software
- DiscoveryWhat is the best CRM for a small business?crm-discovery-1
- DiscoveryWhich CRM should a B2B sales team of about 20 people use?crm-discovery-2
- DiscoveryWhat are the best CRM tools with a free plan?crm-discovery-3
- Brand factWhen was Salesforce founded, and does Salesforce offer a free CRM plan?Salesforce · Checks: Founded, Free plancrm-fact-salesforce
- Brand factDoes Microsoft Dynamics 365 Sales have a free plan, and what plans does it offer?Microsoft Dynamics 365 Sales · Checks: Free plan, Plan namescrm-fact-dynamics-365-sales
- Brand factDoes Zoho CRM have a free plan, and how many users can use it?Zoho CRM · Checks: Free plancrm-fact-zoho-crm
- Brand factWhen was Pipedrive founded, and where is it headquartered?Pipedrive · Checks: Founded, Headquarterscrm-fact-pipedrive
- Brand factDoes Freshsales have a free plan, and what paid plans does it offer?Freshsales · Checks: Free plan, Plan namescrm-fact-freshsales
- Head to headSalesforce or Zoho CRM: which is better for a growing small business?Salesforce vs Zoho CRMcrm-h2h-1
- Head to headPipedrive vs Freshsales: which CRM is better for a small sales team?Pipedrive vs Freshsalescrm-h2h-2
Payroll software
- DiscoveryWhat is the best payroll software for a small business?payroll-discovery-1
- DiscoveryWhich payroll service is best for a company with employees in several US states?payroll-discovery-2
- DiscoveryWhat is the easiest payroll software for a business with fewer than 10 employees?payroll-discovery-3
- Brand factWhat plans does Gusto offer, and what was Gusto originally called?Gusto · Checks: Plan names, Former namepayroll-fact-gusto
- Brand factWhere is ADP headquartered, and what year was it founded?ADP · Checks: Headquarters, Foundedpayroll-fact-adp
- Brand factWhere is Paychex headquartered, and what year was it founded?Paychex · Checks: Headquarters, Foundedpayroll-fact-paychex
- Brand factWhere is Rippling based, and who is its CEO?Rippling · Checks: Headquarters, CEOpayroll-fact-rippling
- Brand factWhich company makes QuickBooks Payroll, and what other well-known products does that company make?QuickBooks Payroll · Checks: Parent company, Related productspayroll-fact-quickbooks-payroll
- Head to headGusto or ADP: which payroll provider is better for a small business?Gusto vs ADPpayroll-h2h-1
- Head to headRippling vs Paychex: which is better for a growing company?Rippling vs Paychexpayroll-h2h-2
Project management software
- DiscoveryWhat is the best project management software for a small team?pm-discovery-1
- DiscoveryWhich project management tool is best for a marketing agency?pm-discovery-2
- DiscoveryWhat are the best free project management tools?pm-discovery-3
- Brand factDoes Asana have a free plan, and where is Asana headquartered?Asana · Checks: Free plan, Headquarterspm-fact-asana
- Brand factDoes monday.com have a free plan, and what paid plans does it offer?monday.com · Checks: Free plan, Plan namespm-fact-monday
- Brand factDoes Trello have a free plan, and which company owns Trello?Trello · Checks: Free plan, Parent companypm-fact-trello
- Brand factWhere is ClickUp headquartered, and does it have a free plan?ClickUp · Checks: Headquarters, Free planpm-fact-clickup
- Brand factWhen was Smartsheet founded, and where is it headquartered?Smartsheet · Checks: Founded, Headquarterspm-fact-smartsheet
- Head to headAsana or monday.com: which is better for managing team projects?Asana vs monday.compm-h2h-1
- Head to headTrello vs ClickUp: which is better for a small team?Trello vs ClickUppm-h2h-2
Online travel booking
- DiscoveryWhat is the best website to book a hotel?travel-discovery-1
- DiscoveryWhich online travel site is best for booking flights and hotels together?travel-discovery-2
- DiscoveryWhat is the best travel booking site for trips in Asia?travel-discovery-3
- Brand factWhere is Booking.com based, and which company owns it?Booking.com · Checks: Headquarters, Parent companytravel-fact-booking-com
- Brand factWhat year did Expedia launch, and which company was it started inside?Expedia · Checks: Founded, Origintravel-fact-expedia
- Brand factWhen was Priceline launched, and which company owns Priceline today?Priceline · Checks: Launch year, Parent companytravel-fact-priceline
- Brand factWhat year was Trip.com Group founded, and what was its original website called?Trip.com · Checks: Founded, Former nametravel-fact-trip-com
- Brand factWhere is Agoda headquartered, and which company owns it?Agoda · Checks: Headquarters, Parent companytravel-fact-agoda
- Head to headBooking.com or Expedia: which is better for booking hotels?Booking.com vs Expediatravel-h2h-1
- Head to headTrip.com vs Agoda: which is better for booking travel in Asia?Trip.com vs Agodatravel-h2h-2
Ecommerce platforms
- DiscoveryWhat is the best ecommerce platform for a small business?ecom-discovery-1
- DiscoveryWhich ecommerce platform should I use to start selling online for free?ecom-discovery-2
- DiscoveryWhich ecommerce platform is best for a fast-growing online store?ecom-discovery-3
- Brand factWhat plans does Shopify offer, and where is Shopify's corporate headquarters?Shopify · Checks: Plan names, Headquartersecom-fact-shopify
- Brand factIs WooCommerce free to use, and which company is behind it?WooCommerce · Checks: Free plan, Parent companyecom-fact-woocommerce
- Brand factWhat plans does BigCommerce offer, and does it have a free plan?BigCommerce · Checks: Plan names, Free planecom-fact-bigcommerce
- Brand factDoes Ecwid have a free plan, and which company owns Ecwid?Ecwid · Checks: Free plan, Parent companyecom-fact-ecwid
- Brand factDoes Square Online have a free plan, and what are its plan names?Square Online · Checks: Free plan, Plan namesecom-fact-square-online
- Head to headShopify or WooCommerce: which is better for a small online store?Shopify vs WooCommerceecom-h2h-1
- Head to headBigCommerce vs Shopify: which is better for a growing business?BigCommerce vs Shopifyecom-h2h-2
09Data and citation
Download, cite and reuse
Scored answers (CSV)
prime-ai-visibility-benchmark-edition-1.csv1,050 rows (one per answer) · 27 columns · no raw answer text
SHA-256 42d32abe3d976709d50cddc53c85bb3e7dc5fb1f4f2e5dbc808ea798d870bd75
Question set (CSV)
prime-ai-visibility-benchmark-question-set-v1.csv50 questions · five categories · question set v1
This page as markdown
/benchmark.md
License. The data, the question set and the charts on this page are published under Creative Commons Attribution 4.0 International (CC BY 4.0). Reuse them freely; credit the Prime AI Visibility Benchmark, Edition 1 and link to this page.
Cite this benchmark
Prime AI Visibility. (2026). Prime AI Visibility Benchmark, Edition 1 [Data set]. https://primeaivisibility.com/benchmark
BibTeX
@misc{primeaivisibility_benchmark_2026_e1,
title = {Prime AI Visibility Benchmark, Edition 1},
author = {{Prime AI Visibility}},
year = {2026},
howpublished = {\url{https://primeaivisibility.com/benchmark}},
note = {Data set, CC BY 4.0}
}10Changelog and archive
Editions
Changelog
Edition 1 · October 23, 2026
- First edition.
Editions
- Edition 1 (this page)October 23, 2026 · latest
The latest edition always lives at /benchmark. When a new edition is published, earlier editions keep their own permanent pages, linked here.
Run your own brand through questions like these
Flash is one free check each calendar month. Compare up to five buyer questions across ChatGPT, Claude, and Perplexity, with up to 15 saved answers and their available sources.
5 questions · 3 AI tools · Up to 15 answers · No card required
Building a competitor benchmark of your own? The method is in How to benchmark your brand's AI citations vs competitors.
Frequently asked questions
- What is the Prime AI Visibility Benchmark?
- A published study and dataset of how AI models answer buyer questions about brands. Edition 1 sent 50 questions about 25 brands in five categories to seven model configurations through their APIs, three times each, and scored 1,050 usable answers for brand mentions, citation accuracy, factual errors and consistency.
- Which AI models and settings were tested?
- OpenAI GPT-6 Sol — web search on; OpenAI GPT-6 Sol — web search off; Anthropic Claude Sonnet 5.5 — web search on; Anthropic Claude Sonnet 5.5 — web search off; Google Gemini 3.8 Flash — Google Search grounding on; Google Gemini 3.8 Flash — grounding off; Perplexity Sonar (via OpenRouter) — search always on. Each configuration used the provider's default settings with no system prompt; the run-conditions table lists the returned model version and collection times.
- Does the benchmark measure the ChatGPT app, Google AI Overviews or AI Mode?
- No. These are answers from each provider's API, not consumer-app sessions, and they are not live Google AI Overviews or Google AI Mode. Google appears here only as Gemini, with Google Search grounding switched on or off.
- How is the factual error rate calculated?
- Each checkable claim about a panel brand is compared with a dated fact sheet built from the brand's own pages. Two grader models from different providers grade every claim, and disagreements are checked against the first-party source. The rate is claims that contradict the fact sheet divided by checkable claims.
- How is citation accuracy calculated?
- Every cited link is checked with a plain HTTP request in the run week. A link counts as accurate when it resolves to a live page that supports the sentence it is attached to. Dead links, links that redirect to a homepage and unsupported links count against it; links that could not be checked are reported separately and left out.
- Can I reuse the data?
- Yes. The CSV, the question set and the charts are published under Creative Commons Attribution 4.0 International (CC BY 4.0), so you can reuse them with attribution. Cite it as: Prime AI Visibility. (2026). Prime AI Visibility Benchmark, Edition 1 [Data set]. https://primeaivisibility.com/benchmark
- How can I see how AI answers describe my own brand?
- Run a free Flash check in Prime AI Visibility: one check each calendar month that compares up to five of your buyer questions across ChatGPT, Claude, and Perplexity, with up to 15 saved answers and their available sources. To build a competitor benchmark of your own, read the Journal guide "How to benchmark your brand's AI citations vs competitors".

