An AI hallucination is a statement from a generative AI system that sounds confident but is false or unsupported, such as an invented court case or a made-up company policy. It happens because these systems predict likely words instead of looking up checked facts. Published tests find errors at very different rates, from about 1 in 5 news answers with major accuracy problems to more than half the answers on hard legal questions. Checking a claim against a primary source is the dependable defence.
01What counts as an AI hallucination?
An AI hallucination is output that a generative AI system presents as true but that is false, invented, or not supported by the material it was given. The US standards agency NIST prefers the word confabulation. It defines this as confidently stated but false content, and notes that people also call it hallucination or fabrication.
In practice it shows up in a few recognisable forms:
- Invented facts, such as a wrong date, a number, or a biography detail that was never true.
- Invented sources, such as a court case, study, quote or web link that does not exist.
- A summary that adds details the original document does not contain.
- A confident explanation of how the answer was reached, when that explanation is also wrong.
NIST warns that the risk grows when people believe false content because the answer sounds sure, and that invented citations can make a wrong answer look checked.
02Why do chatbots sound sure when they are wrong?
NIST explains that these models produce text that matches the statistical patterns in their training data, predicting the next word again and again. That process can give accurate answers and also inaccurate ones. NIST adds that the problem is sharper for long, open-ended answers and for fields that need specialist knowledge (NIST AI 600-1).
A second explanation concerns how models are trained and scored. In a September 2025 paper, Kalai and colleagues argue that common tests reward a confident guess over an honest "I am not sure", so models learn to guess. They propose changing how those tests score uncertainty. That is a proposal, not a finished fix.
The result is that a chatbot gives you no built-in warning when it is guessing. Stanford researchers who tested legal questions found that models often failed to push back on a wrong assumption in the question and could not reliably tell when they were hallucinating (Large Legal Fictions).
A smooth, well-formatted answer is not evidence that the answer is correct.
03How often did published tests find it?
There is no single rate. Results depend on the model version, the task, the wording of the question and the date. The numbers below also measure different things. The EBU count includes any significant issue, such as weak sourcing, so its 45% is not a rate of invented facts. The legal tests asked hard, specific questions that are easy to grade. Here are three measured results, each from one specific test:
- Legal questions, general chatbots: error rates of roughly 58% to 88% depending on the model, on specific questions about US federal court cases (Dahl and others, early 2024).
- Legal research products built to cite real sources: hallucinated answers between about 17% and 33% in a preregistered test, fewer than general chatbots but not zero (Magesh and others, 2024).
- News questions: journalists from 22 public-service media organisations in 18 countries reviewed more than 3,000 answers from four assistants. They found 45% had at least one significant issue, 31% had serious sourcing problems and 20% had major accuracy problems, including invented details and outdated information (EBU and BBC, 22 October 2025).
- Each figure describes the tools tested at that time. None is a current rate for all chatbots.
The EBU release says the BBC's earlier results showed some improvement but error levels stayed high.
04Documented cases where it mattered
In Mata v. Avianca (US District Court, Southern District of New York), lawyers filed a brief citing court opinions that did not exist, produced by ChatGPT. The court's June 2023 order records that a lawyer had asked the chatbot whether one of the cases was real, and it replied that it was. The court found bad faith by the individual lawyers and ordered a $5,000 penalty (court order).
In Moffatt v. Air Canada (British Columbia Civil Resolution Tribunal, 2024 BCCRT 149), a customer relied on an airline website chatbot's statement about bereavement fares. The tribunal rejected the airline's argument that the chatbot was a separate entity and held the airline responsible for its website's output (McCarthy Tétrault summary).
Researcher Damien Charlotin keeps a public database of court and tribunal decisions where AI-fabricated content was found or alleged. It listed 2,207 decisions when we read it on 24 October 2026 and describes itself as a work in progress (database). Our separate page of documented examples goes through more cases.
05What reduces hallucinations, and what does not
One common approach is retrieval: the tool looks up documents first and answers from them. The legal products in the Stanford test worked this way, and they still got a meaningful share of answers wrong (Magesh and others).
Anthropic's developer guidance, one company's published advice that we have not tested, lists techniques such as giving the model permission to say it does not know, asking it to quote the supplied documents word for word before it analyses them, and requiring a supporting quote for each claim. The same page says these steps greatly reduce hallucinations but do not eliminate them (Anthropic documentation).
As of 24 October 2026, we found no source showing that any method removes the problem completely. Treat every answer as a draft.
06How to check an answer before you rely on it
Take the most care where a wrong answer has consequences. NIST singles out consequential decisions and gives healthcare as an example, where a false summary of a patient record could lead to a wrong diagnosis or treatment (NIST AI 600-1).
These steps take a few minutes and work with any chatbot:
- Pick out the claims that can be checked: names, dates, numbers, quotes, citations and links.
- Find each one in a primary source that you open yourself, such as the law text, the court's own website, the company's policy page or the study.
- Search for exact case names, titles and quotes. If you cannot find the item outside the chatbot, treat it as unverified.
- Do not ask the same chatbot whether it was right. In Mata v. Avianca the chatbot assured the lawyer that an invented case was real.
- For law, health, money or safety decisions, ask a qualified person. This page is general information, not legal or medical advice; ask a lawyer or a doctor.
A link that opens does not prove the page says what the chatbot claims. Read the page itself.
?What is not known
- No single, current error rate covers all chatbots. The figures above come from tests of specific model versions, and products change often.
- Whether the scoring change proposed by Kalai and colleagues reduces hallucinations in real products is not yet shown in the sources we read.
- The court database counts decisions that researchers found. How many fabricated filings never reach a written decision is unknown.
QQuestions people ask
Is a hallucination the same as lying?
The sources we read describe it as a by-product of how models generate text, not as an intent to deceive. The effect on the reader can still be harmful, which is why checking matters.
Do AI tools that show their sources still make things up?
Yes. The Stanford test of legal research tools that cite sources still found hallucinations in about 17% to 33% of answers. A citation also has to be opened and read.
Are chatbots getting better at this?
Some measures improve. The EBU release says the BBC's earlier results showed some improvement, but error levels stayed high. Results vary by model and task, so keep checking.
Last checked October 24, 2026. Rules and settings change, so open the source to see the current version.
- NIST AI 600-1, Generative AI Profile National Institute of Standards and Technology NIST's July 2024 profile; we used its definition of confabulation and its explanation of why models produce false content.
- Why Language Models Hallucinate arXiv (Kalai, Nachum, Vempala, Zhang) A September 2025 paper arguing that training and scoring methods reward guessing; we used its explanation and its proposal.
- Large Legal Fictions arXiv (Dahl, Magesh, Suzgun, Ho) A Stanford study of legal errors in general chatbots; we used its error range and its finding on false premises.
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools arXiv (Magesh and others) A preregistered test of commercial legal research tools; we used the error range and the point that retrieval does not remove the problem.
- News Integrity in AI Assistants press release European Broadcasting Union The 22 October 2025 release on a study led by the BBC and EBU; we used its headline percentages.
- Mata v. Avianca, Opinion and Order on Sanctions US District Court, Southern District of New York The court's 22 June 2023 order; we used its findings about the fake opinions and the penalty.
- Moffatt v. Air Canada (2024 BCCRT 149) commentary McCarthy Tétrault LLP A law firm summary of the tribunal decision; we used it for the facts and the holding.
- AI Hallucination Cases database Damien Charlotin A public tracker of court decisions involving fabricated AI content; we used its description and its case count as read on 24 October 2026.
- Reduce hallucinations Anthropic A company's developer guidance; we used its list of techniques and its statement that they reduce but do not eliminate errors.
General information, not legal, medical, financial or investment advice. We are not affiliated with any company, agency or standards body named here. See our editorial policy and corrections, or tell us about a mistake at [email protected].