AI knowledge management: why Copilot gets it wrong

AI knowledge management is the work of making sure that the knowledge AI assistants and agents answer from is accurate, current, owned and permissioned, so that people can rely on their answers. When Microsoft 365 Copilot or an AI agent gives a wrong answer about your company, look first at the document it found: in peer-reviewed tests published in 2026, language models accepted a false recommendation placed in a real hospital document in 46% of cases. The model is the part you buy; the knowledge it reads is the part you can fix.

This guide is for AI and data leads, IT teams rolling out Copilot or an agent, and the knowledge manager they call when the answers go wrong. The term also has a second meaning: AI features inside knowledge management tools. Our comparison of AI knowledge base software covers those. This page is about the knowledge AI depends on.

Grounding AI: why Copilot gives wrong answers

An assistant at work rarely answers from its training alone. Copilot, ChatGPT connected to company files and in-house agents first retrieve passages from the organisation's own content, then write an answer from them. This is called retrieval-augmented generation (RAG), and answering from retrieved sources is called grounding. RAG was designed in 2020 to give answers a traceable source and to let knowledge be updated without retraining the model. It works, but it moves the weak point: an answer is now only as good as the passages retrieved for it.

Several peer-reviewed studies published in 2026 measured what happens when those passages are wrong.

Study	What was tested	What it found
Omar et al., The Lancet Digital Health, January 2026	20 language models, more than 3.4 million prompts, each containing false health information	In plainly worded prompts, the models accepted the false claim 31.7% of the time: 46.1% when it sat in a real hospital discharge note, 8.9% when it came from social media
Mo et al., Findings of ACL 2026	Nine models given medical evidence that had been altered on purpose	The models trusted the altered evidence and answered with confidence, even when it was dangerous
Wallat, Nejdl and Sikdar, Findings of ACL 2026	Facts that changed recently, compared with facts that stay stable	Larger models notice that a fact has changed, but their answers do not change with it
Yuan et al., ACL 2026	1,814 real questions with about 10 retrieved web passages each, 57% of them with passages that contradict each other	The best model scored 0.466 at sorting the conflicting answers; people scored 0.69

The Lancet result matters most for companies. False content was accepted about five times more often when it sat in a real clinical document than when it came from social media. The authors conclude that models absorb fabrications "especially when phrased in authoritative clinical prose". Procedures, policies and product sheets look official in the same way.

Outdated documents

In the newest test, researchers at the University of Southern Denmark built 317 cases where the right answer had changed, in medicine, law, software and platform rules (preprint, September 2026, not yet peer-reviewed). One outdated document overturned 30% to 37% of the answers that two open models got right without it, and 66% to 75% when the models were told to follow the document. When the current document was supplied instead, the models followed it 97% to 100% of the time. Dates alone helped only a little. Saying explicitly when the old guidance stopped applying let large models switch to the current answer almost every time.

That is a knowledge management finding. A date on a document is not enough. A replaced document should say plainly that it no longer applies, and from which date.

Missing content

A 2024 study of three real RAG systems put "missing content" first among seven failure points: when the answer is not in the documents, the system can still produce one. It concluded that such a system can only be fully validated once people use it.

Wide adoption, little measured value

Two recent surveys, built in different ways, find the same thing: AI is widely used, but most firms do not yet see an effect on their results.

Survey	Sample	What it measures	Finding
Firm Data on AI, National Bureau of Economic Research (NBER) working paper 34836, March 2026 version, not peer-reviewed	Nearly 6,000 senior executives in the United States, the United Kingdom, Germany and Australia, November 2025 to January 2026	Executives' own judgement of AI's effect on their firm over three years	69% of firms use AI; 89% report no impact on labour productivity
The state of AI in 2026, McKinsey, August 2026	1,719 online respondents in 97 countries, May to June 2026	Self-reported effect of AI on earnings before interest and taxes (EBIT)	37% attribute at least some EBIT impact to AI, about the same share as a year earlier

Official statistics give the scale. In 2025, 20.0% of EU enterprises with 10 or more employees used at least one AI technology. Among those that had considered AI but did not use it, 43.5% cited difficulties with the availability or quality of the data they needed (Eurostat).

None of these sources says why value is missing. None measures how often deployed assistants fail because their knowledge is wrong, missing or out of date, and no study we read measures a failure rate for AI projects. Read the 89% as executives' judgement about the past three years, not as a failure rate. Our article on why AI projects fail looks at the knowledge side of that question.

What AI reads in a company

Assistants used at work typically read four kinds of content. Each one carries a different risk.

Source	What the assistant gets from it	What can go wrong
Documents in shared drives and SharePoint	Procedures, policies, offers, specifications	Several versions of one document, and nothing to say which one is current
Email, chat and meeting notes	Decisions, context, recent changes	Statements nobody confirmed; personal data
Tickets and case histories	How one problem was solved once	A fix for one case read as a rule for all
A knowledge base	Articles written to be reused	Only as reliable as its owners and its review dates

Microsoft 365 Copilot is the clearest current example of an assistant that reads a team's files. Microsoft calls the layer behind it Work IQ, which its documentation describes as a "workplace intelligence layer". Its developer documentation lists what it can use: emails, meetings and calendars, documents in OneDrive and SharePoint, Teams messages, people and the organisation's structure, Planner plans and enterprise search results. Every request runs as the person who is signed in, so an answer only uses what that person can already open. The documentation explains how Work IQ applies permissions, sensitivity labels and compliance policies. It does not describe any check on whether that content is still true. Our guide to Work IQ goes through each point.

On 14 October 2026, Microsoft announced a team knowledge base in SharePoint, in preview for tenants in Microsoft's Frontier early-access programme: Copilot writes and organises linked pages from the emails, messages, meetings and files a team chooses. Microsoft's own announcement says that "Copilot and agents perform best on well-governed content." A page written by Copilot is still a new document, so someone has to own it, check it and correct it. Our article on Microsoft's team knowledge base explains the feature, and our guide to SharePoint for knowledge management helps you decide whether to supplement SharePoint or replace it.

Permissions follow the person who asks, and they say nothing about quality. They are also only as good as their settings: if a SharePoint site is open to the whole company by mistake, an assistant will use its content in its answers.

The four properties of AI-ready knowledge

Knowledge is ready for AI when it is accurate, current, owned and permissioned. None of these is new to knowledge management. AI makes the cost of a missing one visible, because an assistant passes on what it finds to everyone who asks.

Property	What it means in practice	What the rules and standards say
Accurate	Each item says what is true today, and conflicting versions are resolved	The EU's General Data Protection Regulation (GDPR) requires personal data to be accurate and kept up to date. The US National Institute of Standards and Technology (NIST) suggests verifying that the data behind RAG answers is grounded.
Current	Each item has a review date, and replaced content names what replaced it	NIST lists training data that becomes "stale or outdated" as a risk specific to AI. Joint guidance from US, UK, Australian and New Zealand security agencies warns of "data drift": over time, the data an AI system receives moves away from the data it was built on.
Owned	A named person or team answers for each item, and someone oversees the system	The AI Act requires deployers of high-risk systems to assign competent human oversight, from December 2027 or August 2028 depending on the system. ISO/IEC 42001 sets requirements for an AI management system.
Permissioned	The assistant sees only what the person asking may see, and its rights are reviewed	ANSSI, France's national cybersecurity agency, asks for need-to-know from the design stage and regular reviews of AI tools' access rights. The same joint guidance says an AI output should be classified at the level of its input data.

The EU AI Act, as amended in July 2026

The AI Act entered into force on 1 August 2024. The Digital Omnibus on AI, Regulation (EU) 2026/1744, amended it. It was signed on 8 July 2026, published in the Official Journal on 24 July and has been in force since 27 July 2026. It moved the dates for high-risk systems. The application dates now read as follows (Article 113 and Article 111 of the consolidated text).

From	What applies
2 February 2025	Definitions, AI literacy (Article 4), prohibited practices
2 August 2025	Rules for general-purpose AI models, governance, penalties
2 August 2026	General application, including the transparency duties of Article 50
2 December 2026	Machine-readable marking of AI-generated content, for systems placed on the market before 2 August 2026; two new prohibitions
2 December 2027	High-risk rules for the uses listed in Annex III, including Articles 10 and 26
2 August 2028	High-risk rules for AI built into regulated products (Annex I)

Three points matter for a company that runs an assistant over its own documents.

It is a deployer, not a provider. The Commission's guidelines for general-purpose AI say that a company that modifies a model becomes a model provider only if the change uses more than a third of the original training compute.
Article 4 applies now, as a duty to take measures. Providers and deployers "shall take measures to support the development of AI literacy" of the people who use AI on their behalf. The Commission's questions and answers say that no certificate is needed.
For an ordinary office assistant, the law is silent on the documents it retrieves. Article 10, on data and data governance, covers the training, validation and testing data of high-risk systems. Article 26 asks deployers of high-risk systems to assign human oversight, to make sure the input data they control is relevant, and to keep logs for at least six months. For an ordinary office assistant, no article sets a rule on the quality of the documents behind its answers. That rule is yours to set.

NIST and ISO/IEC 42001

In the United States, the NIST AI Risk Management Framework (version 1.0, January 2023) is voluntary, and NIST says it is being revised. It names training data that may become stale or outdated among the risks that are new or larger for AI than for other software. Its Generative AI Profile (NIST AI 600-1, July 2024) suggests documenting where data comes from, verifying that RAG data is grounded, and assessing the risks again after adding RAG.

ISO/IEC 42001:2023 sets the requirements for an AI management system, for any organisation that provides or uses AI. It is voluntary unless a customer or a contract asks for it. ISO's catalogue shows no revision under way.

Need-to-know: ANSSI and the joint security guidance

France's national cybersecurity agency, ANSSI, published 35 security recommendations for generative AI systems in April 2024 (in French), and still lists them as current. It asks for need-to-know from the design stage: people and tools see only what their work requires. It asks for authorisation checks on every flow between the AI system and business applications. AI tools' access to email, document spaces and code should be reviewed when they are switched on, then regularly, for example every month. ANSSI also says a generative AI system should not take critical decisions without human control.

The joint guidance on AI data security of May 2025, from agencies in the United States, the United Kingdom, Australia and New Zealand, asks organisations to track where data comes from, classify it and control access to it. It adds that an AI output should be classified at the same level as its input data. France's data protection authority, the CNIL, tells organisations to check outputs and prefer systems that cite their sources (2024, in French).

None of these texts says what to do when one answer combines documents with different access rights, or when a right is withdrawn after a document was indexed.

How AI agents connect to knowledge

Agents commonly reach company knowledge through the Model Context Protocol (MCP), an open standard for connecting AI applications to outside systems. In plain words: the AI application (Claude, ChatGPT, Copilot Studio) is the host; the system that holds the knowledge runs a server; the server lets the application read content and use tools. The specification is versioned by date, and its current version is 2026-07-28.

MCP is no longer one company's project. Since 9 December 2025 it has been hosted by the Linux Foundation, inside its new Agentic AI Foundation, with technical decisions taken by named maintainers rather than by companies holding seats. The Linux Foundation's announcement put the number of published MCP servers at more than 10,000.

Three points matter to whoever owns the knowledge:

Consent is built in, security is not. Applications must ask the user before they call a tool or share the user's data. The specification also says that MCP itself "cannot enforce these security principles at the protocol level": each implementation has to.
Authorisation is optional. Where it is used over the web, the specification says it should follow OAuth 2.1, and clients should ask for the least access they need.
Per-document permissions are left to the source. The specification works with access tokens and scopes. It does not describe how a server should filter what it returns to match each person's rights in the source system. That job belongs to the system that holds the knowledge.

Formal standards are still being written. The Internet Engineering Task Force (IETF) set up a working group on agent communication, with milestones from December 2026 to March 2028. NIST launched an AI Agent Standards Initiative on 17 February 2026, with work on agent security and identity. In France, the CNIL published an exploratory note on agentic AI on 20 July 2026: the GDPR fully applies to agents, and the note suggests human validation for the most critical decisions, a separate memory per agent and a way for the user to stop the agent.

What to do, in order

The sources agree on the direction, not on the detail. This order puts the cheapest checks first.

Audit what the assistant answers today. Collect 20 to 30 questions people really ask, with the answer an expert would give. Ask the assistant each one, open the source behind each answer, and mark it right, out of date, wrong, missing or not allowed for that person. This is your baseline, and it shows which domains to fix first. Our AI readiness score takes about three minutes and helps you decide where to start.
Give every domain an owner and every document a review date. When a document replaces another, say so in both, with the date the old one stopped applying: the stale-document study above found that stating this date let large models switch to the current answer almost every time.
Check permissions before the assistant reads anything. Look for sites and folders open to everyone by mistake. Review what the assistant may read when you switch it on, then regularly, as ANSSI advises. Decide what level of confidentiality an answer carries when it draws on several documents.
Keep one governed source per domain. Remove duplicates, archive what is no longer true, and mark which version is authoritative. Models handle contradictions poorly, so have fewer of them.
Test before rollout, then keep testing. Run the same questions again after each change to the content, the model or the connection, and compare. The real test happens in use.

Three questions the sources leave open

None of the sources read for this guide answers these three questions.

How often content should be reviewed. No regulator or standard we read sets a review cycle for the documents an assistant answers from. Public bodies differ: the website of the National Health Service (NHS) in England reviews all its editorial content at least every three years, while GOV.UK's guidance asks teams to monitor content regularly and lets editors set a review date, without giving any frequency.
Who owns content when people and teams change, and how permissions follow an answer built from documents with different access rights.
How to measure whether a knowledge base is accurate. Research says a RAG system can only be fully validated in use, and NIST suggests testing and evaluation. No public body gives metrics or thresholds for an assistant that answers from internal knowledge.

Until a standard answers them, write your own rule for each, and apply it the same way in every domain.

How we work at Elium

Elium is a European knowledge management platform, and these three gaps are where we work with our customers. In Elium, every article has an owner, every space has administrators, and every article can carry a verification date: when the date comes close, the article appears in its owner's task list. An approval flow can stand between a draft and the published version. AI agents connect through MCP, and every call runs with the permissions of the person signed in, so an agent only retrieves what that person could already open.

At the Département de l'Orne, a French local authority, 25 IT professionals give every procedure an expiry date and review it every year, so 100% of their procedures are current. Their technicians get AI answers from those procedures directly inside their support tickets.

FAQ

Common questions

AI knowledge management is the work of making sure that the knowledge AI assistants and agents answer from is accurate, current, owned and permissioned. It covers the documents, articles and records an assistant retrieves, the people who own them, their review dates and who may see them. The phrase is also used for AI features inside knowledge management tools.

AI is used in two directions. Assistants and agents answer questions from an organisation's knowledge, and knowledge management tools use AI to search, summarise, tag and spot duplicates or out-of-date content.

Copilot answers from the content it retrieves, so a wrong, outdated or conflicting document can produce a wrong answer. It draws on what the signed-in person can open in Microsoft 365, and Microsoft's documentation describes no check on whether that content is still true.

Grounding means making an AI model answer from a given set of sources, such as a company's documents, rather than from what it learned in training alone. Retrieval-augmented generation (RAG) is the usual way to do it. Grounding makes answers traceable, but they are only as good as the sources retrieved.

For the documents an assistant reads, AI-ready means four things: the content is accurate, it is current, it has a named owner, and its access rights are correct.

Yes, as a deployer. Article 4 already asks every deployer to take measures for the AI literacy of its staff. The high-risk rules apply only to high-risk uses, from 2 December 2027 or 2 August 2028, and no article sets a quality rule for the documents an office assistant retrieves.

Yes, when permissions are set wrongly. Copilot answers as the signed-in person, so it can use anything that person can open, including a SharePoint site opened to the whole company by mistake. To prevent oversharing, look for sites and folders open to everyone. Review what the assistant may read when you enable it, and then regularly. Decide what level of confidentiality an answer carries when it draws on several documents. Our article on headless knowledge management covers answers that combine documents with different access rights.

The Model Context Protocol (MCP) is an open standard, hosted by the Linux Foundation, that lets AI applications connect to outside systems. It requires user consent before tools are used, but its authorisation section is optional and its specification says it cannot enforce security by itself. Security depends on how each server checks identity and permissions.

No regulator or standard sets a frequency, so give every document an owner and a review date, sooner for content that changes fast. Our guide to keeping a knowledge base accurate explains review intervals and who reviews.

Where to go next

Compare tools: AI knowledge base software compared and enterprise knowledge management software.
Test your knowledge: the AI readiness score, 15 questions.
Do it in Elium: connect ChatGPT, Claude or Copilot Studio to your platform, or see how Mistral's Le Chat, Dust and Glean connect.

Sources

Each source was read between 9 and 15 October 2026, and every link was checked again before publication.

Lewis et al., Retrieval-augmented generation for knowledge-intensive NLP tasks, NeurIPS 2020.
Omar et al., Mapping the susceptibility of large language models to medical misinformation across clinical notes and social media, The Lancet Digital Health 8(1), January 2026 (abstract read on PubMed).
Mo et al., Faithfulness vs. safety, Findings of ACL 2026.
Wallat, Nejdl and Sikdar, When facts change: temporal knowledge conflict resolution in LLMs, Findings of ACL 2026.
Yuan et al., Benchmarking LLM's capability in reasoning over conflicting web references, ACL 2026.
Ahmed, Galke Poech and Röttger, Stale-document poisoning: when outdated retrieval overrides correct model answers, preprint, 25 September 2026.
Barnett et al., Seven failure points when engineering a retrieval augmented generation system, CAIN 2024.
Yotzov, Barrero, Bloom, Bunn, Davis et al., Firm data on AI, NBER Working Paper 34836 (NBER page), version of 16 March 2026.
McKinsey, The state of AI in 2026: on the road to ROI, August 2026.
Eurostat, news release of 11 December 2025 and database isoc_eb_ai, 2025 data.
Microsoft, Work IQ API overview, and Introducing a team knowledge base curated by Copilot and agents, 14 October 2026.
European Union, Regulation (EU) 2026/1744 (Digital Omnibus on AI), and the consolidated AI Act on the AI Act Service Desk: Articles 4, 10, 26, 111 and 113.
European Commission, Guidelines for providers of general-purpose AI models (last updated 11 November 2025) and AI literacy: questions and answers.
NIST, AI Risk Management Framework (AI 100-1, January 2023), Generative AI Profile (AI 600-1, July 2024) and AI Agent Standards Initiative, 17 February 2026.
ISO/IEC, ISO/IEC 42001:2023, AI management system.
ANSSI, Recommandations de sécurité pour un système d'IA générative, version 1.0, 29 April 2024.
NSA, CISA, FBI, ASD's ACSC, NCSC-NZ and NCSC-UK, AI data security, 22 May 2025.
CNIL, questions and answers on using a generative AI system, 18 July 2024, and, with the CIANum, note on agentic AI and personal data, 20 July 2026.
Model Context Protocol, specification 2026-07-28, authorization and governance; Linux Foundation, formation of the Agentic AI Foundation, 9 December 2025.
IETF, Agent Communication Protocols working group.
NHS, website content policy; Government Digital Service, Manage existing GOV.UK content.

Data and sources

This guide is built from public sources only: laws, regulators, standards bodies, public statistics, peer-reviewed research and the documentation of the products it describes. We read about 70 sources between 9 and 15 October 2026 and kept those listed above. Two limits apply. The Lancet Digital Health figures come from the publisher's abstract, because the full text was not available to us. The stale-document study is a preprint, labelled as such where it is quoted. Elium's capabilities are described from our product documentation. If a fact here is wrong or out of date, tell us and we will correct it.

AI knowledge management: why Copilot gets it wrong, and how to fix the source