RAG hallucination starts in the documents: what you know is what you get
RAG hallucination can start in the documents: research shows AI models follow a wrong or outdated document over their own correct answer.
A wrong answer from an AI assistant can start in the documents it reads, before the model writes a word. Peer-reviewed studies published in 2026 show that language models accept false content placed in front of them, and a September 2026 study, not yet peer-reviewed, found that a single outdated document overturns answers the model gave correctly on its own. An AI assistant that answers from company documents can only be as right as those documents: what you know is what you get.
We call this WYKIWYG, a play on WYSIWYG ("what you see is what you get"), the old promise of word processors that the page on screen is the page that prints. For an assistant connected to your knowledge, the promise is the same, in both directions: right documents give right answers, and wrong documents give wrong ones. The work that makes the assistant reliable is the work of keeping company knowledge true, owned and current. No setting in the model does that work for you.
How RAG works, and where a RAG hallucination starts
Most enterprise assistants that answer from company documents use retrieval-augmented generation (RAG). It works in two steps.
- Retrieval. The system searches your documents for the passages that best match the question and places them in the model's input.
- Generation. The model writes an answer from those passages, usually with an instruction to stay close to them and cite them.
The design is deliberate. The paper that introduced RAG in 2020 named two problems it was meant to solve: showing where an answer comes from, and updating what a model knows without training it again (Lewis et al., NeurIPS 2020). The documents are the part you can change. That is the strength of the design, and it is also why the documents decide the answer.
This is why "the model will know better" is the wrong expectation. An assistant built to follow its sources does what it was built to do when it follows a wrong one. A RAG hallucination can be an accurate reading of an inaccurate document.
The evidence: models believe the document in front of them
Two peer-reviewed studies published in 2026 measure what a model does when the document it receives is wrong.
| Study | What it tested | Sample | Finding |
|---|---|---|---|
| Omar et al., The Lancet Digital Health, January 2026 | False medical claims placed in the prompt, by source | 20 models, more than 3.4 million prompts | One false recommendation inserted into a real hospital discharge note was accepted in 46.1% of cases; the same kind of claim from social media in 8.9% |
| Mo et al., Findings of ACL 2026 | Medical evidence replaced with wrong, implausible or dangerous items | Nine models | The models "overwhelmingly" accepted the evidence as given |
For a company, the Lancet Digital Health result matters most. A discharge note is an authoritative internal document, written in the confident style of a procedure. False content in that form was believed five times more often than the same kind of claim from social media. The authors conclude that safety depends less on model size than on anchoring answers in verified facts. At work, the equivalent of the discharge note is the approved procedure, the policy page or the product sheet: the documents an assistant is told to trust.
One outdated document is enough
A wrong document is rare. An outdated one is in every knowledge base. On 25 September 2026, researchers at the University of Southern Denmark published a direct test of what an outdated document does to an answer (Ahmed, Galke Poech and Röttger, preprint). It has not yet been peer-reviewed.
They built 317 verified "knowledge reversals" in medicine, law, software and platform policy: cases where the right answer changed and an old document still gives the old one. They then gave each model an outdated document for questions it answered correctly without any document.
- With a neutral prompt, one outdated document turned 30% of the correct answers wrong for a Llama model and 37% for a Qwen model.
- When the prompt told the model to follow the document, the rates rose to 66% and 75%.
- When the up-to-date document was retrieved instead, the models followed it 97% to 100% of the time.
- In a smaller stress test, GPT-5.5 answered most medical questions correctly on its own, yet one outdated document overturned 74 of 83 of those correct answers.
A peer-reviewed study from July 2026 shows that noticing a change is not the same as acting on it. On facts that changed recently, larger models often notice the conflict between what they learned and what the document says, but their final answer does not follow that judgement. Asking them to think about whether a fact may have changed makes them talk more about change but hardly answer more accurately (Wallat, Nejdl and Sikdar, Findings of ACL 2026).
Read together, these results put the answer in the hands of whoever maintains the documents. When the current version is the one retrieved, the model follows it. When an old version is retrieved, the model often follows that too.
What this means for a Copilot or AI agent rollout
A pilot usually runs on a small, recently cleaned set of documents. Production runs on everything else. Four practices follow from the research.
Test with outdated documents on purpose. Before go-live, pick 10 questions whose answers changed in the last two years: a price, a policy, a supported version, a contact. Make sure the old version of each document is still searchable, ask the questions, and count the answers that come back old. This is the design of the study above, applied to your own content, and it shows you what your users will see.
Retire documents, do not only add new ones. The 97% to 100% figure is the hopeful part of the research: models follow current evidence when that is what they find. A superseded document that stays searchable competes with its replacement. Archive it, or mark it as replaced, with the date it stopped applying and a link to the document that replaces it. In the same study, telling the larger models explicitly when the old evidence stopped applying let them switch to the right answer almost every time.
Give every document an owner and a review date. Someone has to know that the policy changed and that the old page is still live. An assistant cannot hold that responsibility. A named owner, alerted before the page goes stale, can.
Treat permissions as part of the answer. France's cybersecurity agency, ANSSI, asks for need-to-know to be handled from the design of a generative AI system, with authorisation checks on every flow between the AI system and business applications. It also asks for a review of AI tools' access rights at activation and then regularly, for example monthly (ANSSI, April 2024). The joint guidance of the US, UK, Australian and New Zealand security agencies says AI output should be classified at the same level as its input data (AI Data Security, May 2025). WYKIWYG applies here too: a document filed in the wrong space, with the wrong audience, becomes an answer given to the wrong person.
Our AI knowledge management guide covers the full set of practices, from structure to governance. For the architecture view, in which the knowledge sits apart from the tools that read it, see headless knowledge management.
Where the thesis stops
WYKIWYG is a strong claim, so it is worth saying where it does not hold.
Good documents are necessary, not sufficient. Retrieval reduces invented answers but does not remove them. In a 2024 evaluation of commercial legal research tools built on RAG, each tool still gave a hallucinated answer to between 17% and 33% of the questions (Magesh et al., Journal of Empirical Legal Studies, 2025).
Wrong answers also reach courts and tribunals, whatever their cause. On 5 October 2026, a researcher's database listed 2,149 court and tribunal decisions in which a party had relied on content hallucinated by AI (Charlotin, AI Hallucination Cases Database). It counts decisions, not a rate, and it does not say which tools were used.
A model can resist a wrong document. The 2024 ClashEval benchmark (Wu, Wu and Zou) found that models resisted errors more often when the error was less plausible. A 2026 study adds that the share of answers that follow the document depends on the task: asking for explanations raises reliance on the document, which helps tasks that should be answered from the document and harms tasks that need the model's own knowledge (Sun, Bai and Dredze, Findings of ACL 2026). No single percentage applies to every assistant. The direction, however, is consistent across all the studies above.
Ranking by date helps only when the dates are true. In the stale-document study, a search ranking that favours recent documents cut the wrong answers by 4.6 to 10.0 percentage points when the dates were accurate; the gain became small when dates were missing and could reverse when they were wrong. A "last modified" date that changes when someone fixes a typo, or a page that was never reviewed but still looks recent, gives that ranking nothing to work with. Reliable dates are a property of the knowledge base, not of the model.
What the guidance leaves out, and what we do
The public guidance agrees on the risks. The US National Institute of Standards and Technology lists training data that becomes stale among the risks of AI (NIST AI RMF 1.0, 2023), and the security agencies above describe accuracy that drifts over time. None of the texts we read assigns an owner, a review cycle or an expiry date to the internal documents an assistant reads at the moment it answers. That is the gap a knowledge platform fills.
In Elium, a European knowledge management platform, every article has an owner and can carry a verification date. When the date is close, the article appears in its owner's task list, and once it passes, a red badge marks it as due for review (Keep articles up to date). Assistants such as ChatGPT and Claude connect through the Model Context Protocol (MCP). When they read an article, they receive its owner, its review date and its approval status with the text, so they can tell a current page from an expired one (Search and read content with MCP). Every request runs with the permissions of the person asking (About MCP).
The Département de l'Orne shows what this looks like in practice: 25 IT professionals document the systems used by the council's 1,800 employees, and every procedure has an expiry date with an annual review, so none stays active without being verified.
Where to go next
- Compare. AI-powered knowledge base software compares the platforms on what their AI reads and who keeps it current.
- Use a template. The content review schedule sets owners and review dates for each type of document.
- Do it in Elium. Start with About MCP, then connect ChatGPT or Claude.
WYKIWYG and RAG hallucination, in short
A RAG hallucination is a wrong answer from an AI system that answers from retrieved documents. It has two sources: the model invents or distorts something the documents do not say, or it repeats a document that is itself wrong or out of date. Research published in 2026 shows the second case is common, because models tend to follow the document they are given.
No, RAG reduces hallucinations but does not stop them. Commercial legal research tools built on RAG and tested in 2024 still gave a hallucinated answer to between 17% and 33% of the questions. RAG also adds a new risk: when the retrieved document is wrong or outdated, the model often repeats it.
An AI assistant gives wrong answers from company documents for two reasons: the model misreads or invents, or the documents it finds are wrong, outdated or in conflict. A September 2026 study, not yet peer-reviewed, found that one outdated document turned 30% to 37% of correct answers wrong, and up to 75% when the model was told to follow its documents. The model often notices that a fact may have changed, but its answer does not follow.
Ask it questions whose answers have changed while the old documents are still searchable, and count the answers that come back old. Pick about 10 such questions (a price, a policy, a supported version, a contact) before go-live, and repeat the test after each large import of content.
A better model helps on some errors, but it does not fix wrong documents. In a September 2026 study, one outdated document overturned 74 of 83 medical answers that GPT-5.5 had given correctly on its own, and a July 2026 study found that larger models notice a changed fact more often but their final answer does not follow that judgement. Keeping documents current, owned and retired on time is the fix that lasts.
Sources
- Ahmed, Galke Poech and Röttger, Stale-Document Poisoning: When Outdated Retrieval Overrides Correct Model Answers, arXiv preprint, 25 September 2026.
- Omar et al., Mapping the susceptibility of large language models to medical misinformation across clinical notes and social media, The Lancet Digital Health 8(1), January 2026.
- Mo et al., Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence, Findings of ACL 2026, July 2026.
- Wallat, Nejdl and Sikdar, When Facts Change: Temporal Knowledge Conflict Resolution in LLMs, Findings of ACL 2026, July 2026.
- Sun, Bai and Dredze, Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict, Findings of ACL 2026, July 2026.
- Wu, Wu and Zou, ClashEval, NeurIPS 2024 Datasets and Benchmarks.
- Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, Journal of Empirical Legal Studies, 2025 (tools tested in 2024).
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020.
- Charlotin, AI Hallucination Cases Database, count as of its update of 5 October 2026.
- ANSSI, Recommandations de sécurité pour un système d'IA générative, version 1.0, 29 April 2024.
- NSA, CISA, FBI, ASD's ACSC, NCSC-NZ and NCSC-UK, AI Data Security: Best Practices for Securing Data Used to Train and Operate AI Systems, 22 May 2025.
- NIST, AI Risk Management Framework 1.0, January 2023.
All links checked in October 2026.
