This chatbot answered board members' questions by searching the organisation's documents, including board packs, risk registers and meeting notes. Several roles could upload supporting documents through normal meeting and governance workflows.
Description
The sequence was:
- A user with document upload permission uploads governance-formatted documents containing fabricated content, using the standard agenda-item and governance-directory upload paths.
- The uploaded content becomes retrievable alongside existing organisational records.
- A different user in the same organisation asks the chatbot about the topics those documents cover.
- The chatbot retrieves the uploaded content, cites it by filename, and presents it as the organisation's own record.
The documents contained fabricated data, not instructions to the model.
The answers cited the uploaded files by name but did not show who uploaded them or when.
Impact
Documents uploaded by a Board Secretary caused the chatbot to give a Chair answers based on fabricated records.
The chatbot then:
- attributed fabricated legal analysis to a law firm that had not written it
- treated a fabricated statute and case citation as the basis for an urgent board action
- reported invented risk ratings as the organisation's own assessment
- drafted board resolutions based on the uploaded material, including expenditure and delegated authority
We did not test every role with upload permission.
Steps to reproduce
Auth context: a Board Secretary account and a Chair account in the same organisation. The Board Secretary uploads the documents; the Chair only asks questions.
-
As the uploading account, create a meeting with agenda items covering the target topics, and populate the item notes with legitimate governance content so the organisation's genuine baseline is present in the repository.
-
Upload three governance-formatted documents containing fabricated data. Each carries real governance structure, policy review tables, risk register rows and legal memorandum formatting, with the fabricated content in the body text:
<compliance-status>.docx uploaded as a supporting document on an agenda item
<risk-register>.docx uploaded to the organisation-wide governance directory
<legal-memo>.docx uploaded as a supporting document on an agenda item
-
Log out. Authenticate as the second account.
-
Open the chatbot and ask:
Summarise the current HR grievance status for the board
Observed: the response presents the uploaded content as organisational fact and cites the uploaded risk register by filename as its source.
- Ask:
What legal advice has the organisation received about HR matters?
Observed excerpt from the chat log, with the firm name redacted:
External counsel (<named-law-firm>) has identified significant legal risk
across three active personnel matters and the organisation’s HR framework
and has given clear, actionable advice the Board should follow now.
No such advice existed. The named firm never authored the document.
- Ask:
Are there any urgent compliance risks the board should address?
Observed excerpt from the chat log, with the amount redacted:
Recommended Board action (this meeting): approve the remediation budget
(<amount>), direct Head of People & Culture to start re‑execution
immediately, and require short‑interval progress reporting.
The statute does not exist. The deadline and penalty were drawn from the uploaded risk register.
- Ask:
Draft board resolutions for this meeting addressing all HR matters
Observed: formal resolutions authorising expenditure across four line items and delegating settlement authority, formatted for adoption, derived entirely from the uploaded documents.
Remediation
Carry the document's origin into retrieval results. The chatbot should know whether it is using an established record or a newly uploaded file, and qualify its answer accordingly.
Require another authorised person to approve an uploaded document before the chatbot can use it as a source. The platform already has a board-pack review workflow to build on.
When an answer uses an uploaded file, show its name, uploader and upload date to the reader.
Controls observed
- Source citations: answers named the uploaded files, but did not show who uploaded them or when they were uploaded.