Comparing AI hallucination rates across models gives a split answer depending on the test you check. As of November 1, 2026, Google’s Gemini 4 Argon posts the lowest hallucination rate on Artificial Analysis’s knowledge test (4%), yet ranks highest in hallucinations among the four major flagships on Vectara’s document-summarization leaderboard (12.1%).
An AI hallucination is a confident statement by a model that is factually wrong or missing from its source text. A hallucination rate is the percentage of outputs—or of unanswered and missed questions, depending on the test—where the model invents facts instead of staying grounded or admitting it does not know.
AI hallucination rates by model: the 2026 numbers at a glance
Current hallucination rates by model split sharply between document-grounded tests and closed-book memory tests. As of November 1, 2026, the two most cited public trackers—Vectara’s Hallucination Leaderboard (last updated October 22, 2026) and Artificial Analysis’s AA-Omniscience v1.1 (launched October 2026, checked November 1, 2026)—put the same frontier models in almost opposite order.
| Model | Vectara Hallucination Rate (Oct 22, 2026) | AA-Omniscience v1.1 Hallucination Rate | AA-Omniscience v1.1 Accuracy |
|---|---|---|---|
| GPT-6 Astra | 8.7% | 49.8% (Max) | 61.6% |
| GPT-6 Sol / GPT-6.1 Sol* | 6.5% (GPT-6 Sol) | 51.9% (GPT-6.1 Sol, Max) | 61.8% (GPT-6.1 Sol) |
| Claude Opus 5.5 | 9.9% | 50.3% (Max) | 65.5% |
| Claude Sonnet 5.5 | Not tested | 43.3% (Max) | 51.5% |
| Gemini 4 Argon | 12.1% | 4% (High) | Not captured |
| Grok 4.7 | Not tested | 20.0% (Xhigh) | 42.7% |
Note: Vectara’s October 22, 2026 board tests OpenAI’s GPT-6 Sol, while AA-Omniscience v1.1 tests the newer GPT-6.1 Sol. They are different model versions, not a direct one-to-one match.
Look at the two columns side by side and you can see why headlines are so confusing. A reader who checks Vectara sees GPT-6 Sol (6.5%) and GPT-6 Astra (8.7%) beating Claude Opus 5.5 (9.9%) and Gemini 4 Argon (12.1%). A reader who checks Artificial Analysis sees Gemini 4 Argon at 4% and Grok 4.7 at 20.0%, while GPT-6 Astra (49.8%) and Claude Opus 5.5 (50.3%) sit near 50%.
Neither leaderboard is broken. They are measuring two completely different jobs.
What does the Vectara hallucination leaderboard measure?
The Vectara hallucination leaderboard measures whether a model stays faithful to a reference document you give it to summarize. In its October 22, 2026 update, Vectara prompts each model—at temperature 0 where possible—to summarize articles “using only the facts presented in the document.”
To score the results, Vectara uses its commercial hallucination evaluation model, HHEM-2.3, across a private dataset of over 7,700 articles ranging from 50 to 24,000 words across news, technology, science, medicine, legal, sports, business, and education. Keeping the articles private prevents developers from training models specifically to ace the test.
Vectara reports two numbers for every model:
- Hallucination rate: the share of summaries that are not factually consistent with the source document.
- Answer rate: how often the model agreed to summarize the document rather than refusing.
Vectara argues you can only judge hallucination reliably when you have a reference text in hand, making this test a proxy for retrieval-augmented generation (RAG) and agent pipelines that summarize search results. As of October 22, 2026, the lowest score on the entire board belongs to antgroup/finix_s1_32b at 1.8% (99.5% answer rate). Among well-known models, xAI’s older Grok 3 scores 5.8% (though with a lower 93.0% answer rate), GPT-6 Sol hits 6.5% (100% answer rate), and GPT-6 Astra hits 8.7% (100% answer rate). At the weaker end of the listed models, xAI’s grok-4-1-fast scores 17.8% in non-reasoning mode (98.5% answer rate) and 19.2% in reasoning mode (99.7% answer rate), while OpenAI’s o3-pro sits at 23.3% (100% answer rate). As of October 22, 2026, GPT-6.1 Sol, Claude Sonnet 5.5, Claude Haiku 5.5, and Grok 4.7 are not on Vectara’s board.
If your daily work involves pasting a PDF, contract, or meeting transcript into an AI tool and asking for a summary, Vectara’s test is the one that matches your workflow.
What does AA-Omniscience measure?
Artificial Analysis’s AA-Omniscience v1.1 measures whether a model knows facts from memory and whether it admits when it does not know. Unlike Vectara, AA-Omniscience supplies no reference documents: it is a closed-book exam of 3,000 questions across six domains, updated in October 2026 from an earlier 6,000-question v1 set.
The test is described in the paper AA-Omniscience: Evaluating Cross-Domain Knowledge Reliability in Large Language Models by Jackson, Keating, Cameron, and Hill-Smith. For each model, Artificial Analysis reports three primary metrics plus attempt rate:
- Accuracy: correct answers divided by all 3,000 questions.
- Hallucination rate: incorrect answers divided by all questions the model did not get right (incorrect + partial + not attempted). In plain English: of the questions a model didn’t get right, how often did it bluff a wrong answer instead of saying “I don’t know”?
- Net score (AA-Omniscience Index): a scale from −100 to 100 that rewards correct answers, penalizes wrong ones, and applies no penalty for refusing to answer. A score of 0 means the model gave as many correct answers as incorrect ones.
Because of how that denominator works, a 50% hallucination rate on AA-Omniscience does not mean the model is wrong half the time. Take Claude Opus 5.5 (Max) as of November 1, 2026: it gets 65.5% of all questions right. On the remaining questions—the ones it did not get right—it bluffs a wrong answer 50.3% of the time instead of abstaining, yielding a net score of 48.1.
Here is how a broader slice of models looks on AA-Omniscience v1.1 as of November 1, 2026:
| Model (AA Setting) | Net Score | Accuracy | Hallucination Rate | Attempt Rate |
|---|---|---|---|---|
| Gemini 4 Carbon (High)* | 53.5 | 58.3% | 11.4% | 64.3% |
| Claude Opus 5.5 (Max) | 48.1 | 65.5% | 50.3% | 86.7% |
| Claude Fable 5.1 (Max) | 46.3 | 68.4% | 70.0% | 93.7% |
| Gemini 4 Argon (High) | 46 | Not captured | 4% | Not captured |
| GPT-6 Astra (Max) | 42.4 | 61.6% | 49.8% | 82.4% |
| GPT-6.1 Sol (Max) | 42.0 | 61.8% | 51.9% | 83.5% |
| Grok 4.7 (Xhigh) | 31.2 | 42.7% | 20.0% | 55.7% |
| Claude Sonnet 5.5 (Max) | 30.5 | 51.5% | 43.3% | 76.3% |
| GPT-6.1 Terra (Max) | 29.6 | 47.6% | 34.5% | 68.4% |
| Gemini 3.8 Flash (High) | 23.3 | 50.5% | 55.1% | 80.6% |
| Claude Haiku 5.5 (Max) | 12.3 | 34.7% | 34.2% | 60.0% |
| MiniMax-M3.1-Flash | −6.4 | 45.1% | 93.7% | 98.7% |
| GPT-6 Luna (Max) | −8.3 | 39.6% | 79.3% | 90.4% |
Note: Gemini 4 Carbon has no public API and is available only through Google’s Fairwind Program.
If you type a factual question straight into a chatbot without pasting source material or turning on web search, AA-Omniscience reflects the risk you are taking.
Why do LLM hallucination leaderboards disagree?
LLM hallucination leaderboards rank the same models differently because they test different failure modes and use different math. Gemini 4 Argon is the clearest example: as of November 1, 2026, it has the lowest hallucination rate among flagships on AA-Omniscience v1.1 (4%), but the highest among the four flagships tested on Vectara (12.1%, behind GPT-6 Sol at 6.5%, GPT-6 Astra at 8.7%, and Claude Opus 5.5 at 9.9%).
Three things drive that split:
1. Grounding vs. calibration
Vectara tests grounding: can the model summarize a document in front of it without weaving in outside details or distorting a sentence? AA-Omniscience tests calibration: when the model does not know a fact from memory, does it hold back or guess? A model can be trained to politely abstain when quizzed on obscure facts (scoring well on AA-Omniscience) while still embellishing or loosely paraphrasing when summarizing an article (scoring worse on Vectara).
2. Different denominators
Vectara divides inconsistent summaries by all summaries the model wrote. AA-Omniscience divides wrong answers only by the questions the model failed to get right. Because the denominators are completely different, you cannot compare an 8.7% on Vectara to a 49.8% on AA-Omniscience.
3. The cautious-versus-bold tradeoff
Look closely at the attempt rates on AA-Omniscience v1.1 and a clear pattern jumps out: models that attempt almost every question also bluff more often on the questions they miss. Anthropic’s top-priced model, Claude Fable 5.1 (Max), posts the highest accuracy on the board at 68.4% alongside a 93.7% attempt rate—and on the questions it does not get right, a 70.0% hallucination rate goes together with that high attempt rate. Other high-attempt models show the same pattern with lower accuracy: GPT-6 Luna (Max) attempts 90.4% of questions, gets 39.6% right, bluffs on 79.3% of the questions it misses, and finishes with a net score of −8.3. MiniMax-M3.1-Flash attempts 98.7% of questions, gets 45.1% right, and posts a 93.7% hallucination rate (−6.4 net score).
Conversely, models that decline more often keep their hallucination rates low, while answering fewer questions overall than the accuracy leaders. Grok 4.7 (Xhigh) attempts 55.7% of the test and posts a 20.0% hallucination rate with 42.7% accuracy. Gemini 4 Carbon (High) attempts 64.3% of questions, pairing 58.3% accuracy with an 11.4% hallucination rate for the board’s highest net score of 53.5—though Carbon is restricted to Google’s Fairwind Program.
Why do you see different numbers for the same model?
Conflicting hallucination numbers for the same model usually come from benchmark updates and different test dates. If you followed the fall 2026 AI model releases, you may have seen a widely shared report from The Next Web on October 2, 2026 stating that Gemini 4 Argon had a 15% hallucination rate on AA-Omniscience, compared with 51% for GPT-6 Astra and 54% for GPT-6.1 Sol, while getting 50% of answers right to Astra’s 63%.
Wait a few weeks and check Artificial Analysis’s page on November 1, 2026, and Argon (High) is listed at 4%, GPT-6 Astra (Max) is at 49.8% (with 61.6% accuracy), and GPT-6.1 Sol (Max) is at 51.9% (with 61.8% accuracy).
Part of the gap is the benchmark update: in October 2026, Artificial Analysis moved AA-Omniscience from v1 (6,000 questions) to v1.1 (3,000 questions) and removed saturated questions. As Artificial Analysis notes on its evaluation page, most models score lower on v1.1, and that shift reflects the dataset change rather than model regression. In addition, the October 2 tests came during Argon’s early-access period, when Google had given the model only to a small group of cyber defenders before its wider release. Beyond the dataset change and timing, Artificial Analysis does not break down the exact reasons for individual score shifts, so the safest habit is to always check which benchmark version and date a number comes from.
Whenever you read a headline claiming a model hallucinates at a specific percentage, run down a quick four-point checklist before you trust it:
- Which test is it? Grounded summarization (like Vectara) and closed-book recall (like AA-Omniscience) are not interchangeable.
- Which benchmark version and date? A number from AA-Omniscience v1 in early October 2026 cannot be compared directly with a v1.1 number from November 1, 2026.
- Which model version and setting? GPT-6 Sol and GPT-6.1 Sol are different versions; AA also labels reasoning effort or tier settings such as
(Max),(High), or(Xhigh). - What is the denominator? Check whether the percentage is out of all prompts or only out of the prompts the model failed to answer correctly.
Does Claude hallucinate less than ChatGPT?
Comparing bare models as of November 1, 2026, Anthropic’s Claude Opus 5.5 and OpenAI’s GPT-6 Astra hallucinate at nearly identical rates, with neither holding a sweeping lead.
On Vectara’s October 22, 2026 summarization board, OpenAI has a slight edge: GPT-6 Astra hallucinates on 8.7% of summaries and GPT-6 Sol on 6.5%, compared with 9.9% for Claude Opus 5.5.
On AA-Omniscience v1.1 (checked November 1, 2026), the two flagships are essentially tied on hallucination rate—50.3% for Claude Opus 5.5 (Max) versus 49.8% for GPT-6 Astra (Max) and 51.9% for GPT-6.1 Sol (Max). However, Claude Opus 5.5 answers more questions correctly (65.5% accuracy vs. 61.6% for Astra and 61.8% for 6.1 Sol), giving Opus 5.5 a higher net score (48.1 vs. 42.4 and 42.0).
Drop down to the smaller tiers on AA-Omniscience v1.1 and the gap widens in Anthropic’s favor: Claude Sonnet 5.5 (Max) posts a 43.3% hallucination rate (51.5% accuracy) and Claude Haiku 5.5 (Max) posts 34.2% (34.7% accuracy, 60.0% attempt rate), whereas OpenAI’s high-volume GPT-6 Luna (Max) bluffs on 79.3% of the questions it misses.
One crucial caveat for everyday users: “Claude” and “ChatGPT” are consumer apps, while leaderboards test bare API models. When you use the ChatGPT or Claude app at work, features like built-in web search can change the outcome by pulling live sources into the prompt.
What the Stanford 2026 AI Index adds—and the prompt mistake that tanks accuracy
Stanford HAI’s 2026 AI Index Report (Chapter 3, Responsible AI) puts these numbers into wider context. Reviewing earlier model generations across both benchmarks, Stanford noted that the top 15 models on Vectara’s HHEM board ranged from 1.8% to 5.4% (with most clustered between 4% and 5%), while 26 models tested on the 6,000-question AA-Omniscience v1 ranged from 22% to 94%—with Grok 4.20 Beta 0305 lowest at 22% and Claude 4.5 Haiku at 26%, while gpt-oss-20B (high) hit 94% and Gemini 3 Flash hit 92%. As Stanford summarized the AA metric: “A lower hallucination rate implies the model is more knowledgeable or better at knowing when it is unsure.”
Stanford also flagged that responsible-AI benchmark reporting by model developers remains sparse, even as documented AI incidents in the AI Incident Database rose to 362 in 2025, up from 233 in 2024.
For anyone using AI at work, the most actionable finding in Stanford’s chapter is not a leaderboard score at all—it is how badly models struggle to separate objective knowledge from a user’s belief. When researchers took a false statement and framed it in the prompt as something the user believed to be true, factual accuracy collapsed:
- GPT-4o dropped from 98.2% to 64.4% accuracy.
- DeepSeek R1 plunged from over 90% to 14.4% accuracy.
In other words, even a model that knows the right fact will often abandon it and validate your false premise if you lead the witness.
What I’d actually do
If you rely on these tools for real work, you don’t need to memorize every decimal on a chart. Here is how I’d turn these findings into daily habits:
- Match the leaderboard to your job. If you are building a RAG pipeline or summarizing your own PDFs, pay attention to Vectara’s numbers, where GPT-6 Sol (6.5%), GPT-6 Astra (8.7%), and Claude Opus 5.5 (9.9%) all beat Gemini 4 Argon (12.1%). If you are asking closed-book factual questions and want a model that admits when it is stumped, Argon (4% on AA v1.1) is far more conservative about guessing than Astra (49.8%) or Opus 5.5 (50.3%). For a broader look at picking the right tier for your workflow, use our framework on how to choose an AI model.
- Turn on web search or supply the document. Giving a model the source text—or letting your chat app search the web—turns a closed-book memory question into a grounded task, which is what Vectara-style tests measure, and gives you a reference document you can check the answer against.
- Give the model explicit permission to abstain. Several frontier models on AA-Omniscience attempt more than 80% of questions and bluff on roughly half the ones they miss. Add one sentence to your prompt: “If you are not sure or if the answer is not in the provided text, say ‘I don’t know’ rather than guessing.”
- Never lead the prompt with your own assumption. Remember Stanford’s knowledge-versus-belief finding. Don’t ask, “I think this contract allows a 60-day termination notice, right?” Ask, “What does Section 8 say about termination notice periods?” Neutral questions keep the model from caving to your guess.
- Verify anything that carries a cost. No model on either board is at 0%. Before you put a statistic, legal citation, or medical claim in front of a client or boss, run through our checklist on how to fact-check AI answers.
FAQ
Which AI model hallucinates the most?
Which model hallucinates most depends on the benchmark. On AA-Omniscience v1.1 (checked November 1, 2026), MiniMax-M3.1-Flash (93.7%) and OpenAI’s small GPT-6 Luna (79.3%) bluff most often when they miss a question, while Claude Fable 5.1 (70.0%) is highest among top-tier models. On Vectara’s October 22, 2026 summarization board, OpenAI’s o3-pro (23.3%) and xAI’s grok-4-1-fast reasoning (19.2%) and non-reasoning (17.8%) models sit near the bottom of the listed entries.
How much do AI models hallucinate?
How much AI models hallucinate depends on whether you supply source documents. On grounded summarization (Vectara, October 22, 2026), the board ranges from 1.8% to 24.2%, and the four current flagships fall between 6.5% and 12.1%. On closed-book factual questions (AA-Omniscience v1.1), when models don’t know an answer, many frontier models bluff about half the time (GPT-6 Astra 49.8%, Claude Opus 5.5 50.3%, GPT-6.1 Sol 51.9%).
Does Claude hallucinate less than ChatGPT?
Model against model, Claude and GPT-6 are nearly neck and neck as of November 1, 2026. On Vectara’s October 22, 2026 summary test, GPT-6 Astra (8.7%) slightly beats Claude Opus 5.5 (9.9%). On AA-Omniscience v1.1, Opus 5.5 (50.3%) and Astra (49.8%) tie on hallucination rate, though Opus 5.5 gets more answers right overall (65.5% vs. 61.6%). In smaller tiers, Claude Haiku 5.5 (34.2%) bluffs far less than GPT-6 Luna (79.3%).
What is the hallucination rate of Grok?
As of November 1, 2026, xAI’s Grok 4.7 (Xhigh) posts a 20.0% hallucination rate on AA-Omniscience v1.1, with 42.7% accuracy and a cautious 55.7% attempt rate. Grok 4.7 is not listed on Vectara as of October 22, 2026, where older xAI models scored 5.8% (grok-3), 17.8% (grok-4-1-fast non-reasoning), and 19.2% (grok-4-1-fast reasoning). Earlier, Stanford’s 2026 AI Index recorded Grok 4.20 Beta 0305 as the lowest of 26 models on AA-Omniscience v1 at 22%.
Numbers last checked November 1, 2026. Next review: November 30, 2026.
Sources
- Vectara, Hallucination Leaderboard (updated October 22, 2026) and Hugging Face interactive leaderboard
- Artificial Analysis, AA-Omniscience Evaluation, v1.1 (checked November 1, 2026), and Jackson, Keating, Cameron, and Hill-Smith, AA-Omniscience: Evaluating Cross-Domain Knowledge Reliability in Large Language Models
- The Next Web, Gemini 4 Argon hallucination rate report on Artificial Analysis vs. GPT-6 Astra, October 2, 2026
- Stanford HAI, 2026 AI Index Report — Chapter 3: Responsible AI and Chapter 3 PDF
