LLM hallucinations: why they happen and how to reduce them

Ask a language model for a statistic, a court case or a library function and it may hand you one that doesn't exist, in the same confident tone it uses for facts. These errors are called hallucinations. This guide explains why they happen, what published measurements show about how often, and the practical steps that reduce, catch and measure them in your own app.

What a hallucination is

A hallucination is output that is fluent and confident but false, or not supported by the information the model was given. NIST's generative AI risk profile calls the same problem confabulation, defined as "the production of confidently stated but erroneous or false content."

It helps to separate two failures. A factuality error contradicts the real world, like a wrong founding date. A faithfulness error contradicts or goes beyond the source you supplied, like a summary that adds a figure the document never mentions.

Both matter, but in apps built on your own documents, faithfulness is the one you can most directly measure and control. Either way, hallucinations are not a bug a patch will remove. They are a side effect of how models produce text, so every serious LLM application needs a plan for them.

Why models hallucinate

Several causes stack together:

• Next-token prediction. A model writes one token at a time by predicting what plausibly comes next. Plausible and true usually overlap, but nothing in that mechanism checks a claim against reality before writing it.
• Gaps in training data. During pretraining, a model sees common facts thousands of times and rare ones once or never. A 2025 paper by Kalai and colleagues argues that when a model can't reliably separate true statements from false ones, statistical pressure produces errors, and rare, arbitrary facts such as an obscure person's birthday are the most exposed.
• Knowledge cutoffs. Training data stops at a date. OpenAI lists April 30, 2026 as the knowledge cutoff for GPT-6 Astra, for example, so without a search tool it can't know about later events and may fill the gap with a guess.
• Decoding. Most apps sample from the model's probability distribution, shaped by settings like temperature and top-p. Sampling can pick a less likely, wrong continuation, and because each token depends on the ones before it, an early wrong detail can carry through the whole answer.
• Pressure to answer. Fine-tuning for helpfulness can reward complete-sounding answers, and Kalai and colleagues argue that accuracy-only benchmarks reward guessing over admitting uncertainty, leaving models "optimized to be good test-takers." Questions with false premises add to the pressure: ask why something happened and a model may explain an event that never occurred.

OpenAI's September 2025 explainer shows the trade-off with numbers from its SimpleQA factual quiz. The older o4-mini answered 24% of questions correctly but was wrong on 75% because it almost never abstained. The newer gpt-5-thinking-mini was slightly less accurate at 22%, but it declined 52% of questions and was wrong on only 26%.

Types of hallucinations

Different types need different defenses, so it's worth naming them.

Citation errors have been measured directly. In a 2023 study, Walters and Wilder found that 55% of the citations GPT-3.5 produced and 18% of those from GPT-4 were fabricated. Even among real references, 43% and 24% respectively contained substantive errors.

Code has its own version. A 2024 study by Spracklen and colleagues generated 576,000 code samples with 16 models and found that, on average, at least 5.2% of the packages suggested by commercial models and 21.7% of those suggested by open-source models did not exist. That is also a security issue, since an attacker could publish a malicious package under a commonly invented name; see LLM security and privacy.

How often LLMs hallucinate

There's no single hallucination rate. It depends on the model, task, domain, whether sources are supplied and how errors are counted. Published, dated measurements show the range:

• Grounded summarization. Vectara's leaderboard, last updated May 11, 2026, asks models to summarize more than 7,700 documents using only facts in the text and scores the results with its own classifier. Rates ranged from 1.8% to 24.2%. Bigger wasn't always better: GPT-5.4 nano scored 3.1% against 8.3% for GPT-5.4 Pro, and Gemini 2.5 Flash-Lite scored 3.3% against 13.6% for Gemini 3 Pro Preview.
• Open factual questions. In OpenAI's 2025 SimpleQA comparison above, error rates ran from 26% to 75%, driven largely by each model's willingness to abstain.
• Specialized retrieval tools. A 2024 study by Magesh and colleagues found that commercial legal research tools built on retrieval made fewer errors than a general-purpose chatbot, yet still hallucinated between 17% and 33% of the time.

Treat these as snapshots, not specifications. The lesson that lasts is that retrieval lowers error rates without removing them, and that your rate on your own data is the only one that matters. For how to read scores like these, see LLM benchmarks explained.

How to reduce hallucinations

No single technique solves the problem. Stack several, starting with whichever removes the most risk for your use case.

Ground answers in your sources

Grounding means giving the model the facts it needs and instructing it to answer only from them. Retrieval-augmented generation (RAG) automates this by searching your documents for each question; the original RAG paper found retrieval made generated text more specific and factual than a model working from memory alone.

Ask for citations a program can check, and say exactly what to do when the sources fall short:

Answer using only the sources below. End each sentence with its source ID, like [S2].
If the sources don't contain the answer, reply exactly:
"I couldn't find this in the provided documents."
If sources disagree, say so and cite both. Do not use outside knowledge.

<sources>
[S1] ...
[S2] ...
</sources>

Question: {question}

For long documents, Anthropic's guidance on reducing hallucinations suggests having the model extract word-for-word quotes first, answer from those quotes, and retract any claim it can't support with one. The prompt library includes grounded-answer templates you can adapt.

Fix retrieval before blaming the model

If the right passage never reaches the prompt, even a careful model will guess or fall back on stale memory. When a grounded answer is wrong, first check whether the retrieved passages contained the answer at all.

Common fixes include better chunking, hybrid keyword and semantic search, a reranking step, metadata filters for dates and versions, and removing outdated duplicates from the index. RAG explained covers each in detail.

Make "I don't know" an acceptable answer

State in the system prompt that saying the information isn't available is a correct response, and give the exact wording to use. Ask the model to flag uncertain claims and to point out false premises instead of answering around them. Then make sure your app handles abstentions well, for example by offering a search or a handoff to a person.

Constrain the output and validate it

Structured outputs force a response to match a JSON schema. OpenAI's documentation describes the guarantee as schema adherence, and a perfectly shaped field can still hold a wrong value, so validate content too: check that IDs exist in your database, dates parse, totals add up and quoted text actually appears in the source.

Make fields nullable and tell the model to return null when information is missing. A required field with no data behind it is an invitation to invent.

Use tools for facts, math and code

With tool calling, the model can look up live data, query a database, use a calculator or run code instead of recalling or estimating. A currency conversion done by a code tool can be checked; one done in the model's head is a guess that looks like arithmetic. For generated code, run it, type-check it and confirm each dependency exists in your package registry before installing anything.

Add a verification pass

A second pass catches errors the first one made. Chain-of-Verification, from a 2023 paper by Dhuliawala and colleagues, drafts an answer, writes fact-checking questions about it, answers them independently and then revises.

Simpler variants work too: a separate checker prompt that compares each claim against the sources, or asking the same question several times and flagging disagreement. A 2024 Nature paper formalized the second idea as semantic entropy, detecting confabulations by measuring how much sampled answers differ in meaning. Verification adds calls and cost, so reserve it for high-stakes outputs.

Choose the model and settings deliberately

Models differ substantially, and rankings change by task, so compare candidates on your own test set rather than a general leaderboard. Extra reasoning doesn't automatically help: on Vectara's leaderboard, the heavier-reasoning settings of several models scored slightly worse than lighter settings of the same model.

Lower temperature makes outputs more consistent, but a model can be consistently wrong, so treat it as a small lever rather than a fix.

Before launch, confirm each of these:

UX patterns that keep users safe

Your interface decides whether an error gets caught or acted on. These patterns help:

• Show sources beside claims. Link each citation to the exact passage, not just the document, so checking takes one click.
• Make "not found" a visible state. A clear "I couldn't find this in your documents" with a next step beats a confident guess.
• Highlight what to verify. Numbers, names, dates, prices and legal or medical references are where errors hurt most, so mark them visually.
• Avoid fake precision. Don't display confidence percentages unless you've checked that they match observed accuracy.
• Keep people in the loop for consequential actions. Let the model draft the email, refund or code change, and let a person approve it.
• Route high-stakes topics. Use guardrails to send medical, legal, financial or safety questions to a person or a vetted source.
• Make errors easy to report. A one-click "this is wrong" button that captures the conversation feeds your test set.
• Set honest expectations. Tell users the assistant can make mistakes, and what it is and isn't for.

How to measure hallucinations in your app

A repeatable evaluation turns "it seems fine" into a number you can improve:

1. Build a test set. Collect 100 to 300 real questions with reference answers and supporting sources. Include questions your sources can't answer and questions with false premises, because that's where models invent.
2. Grade three outcomes. Following the SimpleQA approach, label each answer correct, incorrect or not attempted. For grounded apps, also check that every claim is supported by its cited passage.
3. Scale grading with a judge model. LLM-as-a-judge grading is fast, but calibrate it against human labels on at least 50 answers, and recheck whenever you change the judge.
4. Rerun on every change. A new model, prompt, chunking strategy or retriever can move the rate in either direction.
5. Monitor production. Review a sample of live conversations each week, track user reports and add confirmed failures to the test set.

Here's how the numbers work in a hypothetical set of 200 questions, 40 of them unanswerable from your documents. The model gets 150 right, answers 18 wrongly and abstains on 32. Its overall error rate is 18 ÷ 200 = 9%, and its abstention rate is 32 ÷ 200 = 16%.

Splitting by question type is more revealing. On the 40 unanswerable questions it abstained 28 times and invented 12 answers, so 30% of them got a made-up response. On the 160 answerable ones it got 150 right, 6 wrong and abstained on 4, an over-caution rate of 2.5%. Drive the 30% down first, and watch the 2.5% to make sure your fixes don't make the assistant unhelpfully timid.

If you'd like help designing an evaluation set or review workflow, you can get hands-on help from LLM specialists.

When some invention is fine

The same tendency that produces a fake citation also produces a fresh plot twist, a product name or twenty headline options. For brainstorming, fiction, naming, role-play, sample dialogues and placeholder data, invention is the point, and a higher temperature often helps.

Keep three boundaries. Label invented material clearly, especially realistic sample data that could leak into real records. Never let the model attribute invented quotes or facts to real people or organizations. And keep creative and factual modes apart: a brainstorming prompt shouldn't share a system prompt with your support assistant.

Common mistakes

• Trusting a citation because it exists. Check that the cited passage actually supports the claim.
• Asking the model whether it's sure. A model asked to double-check itself in the same conversation can simply restate its error. Verify against sources or with a separate checker prompt.
• Testing only answerable questions. Your test set needs questions the model should decline.
• Blaming the model for retrieval failures. Inspect the retrieved passages before switching models.
• Treating temperature 0 as a fix. It reduces randomness, not error.
• Choosing a model from a general leaderboard. Rankings shift across tasks, so test on your own data.
• Taking an agent's report at face value. Confirm actions in tool logs; AI agents explained covers why.

Next steps

• Build an assistant that answers from your documents and cites them with RAG explained.
• Write tighter instructions, including abstention rules and output formats, with the prompt builder.
• Shortlist models to compare on your own test set with the LLM picker.