Why our AI cites clauses instead of sounding confident
Design notes on making AI findings verifiable — every verdict is tied to a clause and cites the passages it relied on, and a person decides what counts.
Early in building StayEvident we made a design decision that felt constraining at the time: every finding the AI produces must be tied to the specific clause or control it is assessing, and must cite the passages of your documents it is drawing on. The alternative, a confident summary that synthesises across your documents, is easier to read and harder to verify.
We chose hard to verify over easy to read because ISO management is an audit-facing activity. If an auditor asks why you closed a gap, the answer cannot be “the AI said so.” It has to be a requirement, a piece of evidence, and a person who decided the one satisfies the other.
These are notes on how that decision shaped the design.
One clause at a time
The first choice was the unit of work. A tempting design is to hand a model your whole document library and ask it for a gap analysis. That produces fluent output, but it is very hard to check: findings drift between clauses, requirements get merged, and when something is missing it is not obvious whether the model looked and found nothing, or never looked.
So an assessment runs per clause. For each in-scope leaf clause of ISO 9001 or ISO/IEC 27001, including each Annex A control, StayEvident retrieves the passages from your library that are most relevant to that requirement, and the AI assesses that clause against those passages alone. Excluded clauses are skipped entirely, so the assessment follows the scope decisions you have already made.
This has a few consequences we like:
- Every verdict is anchored to a clause by construction. There is no finding that floats free of the standard.
- Coverage is explicit. You can see which clauses have been assessed and which haven’t.
- When a verdict is wrong, it is wrong about one clause, and you can see exactly what the AI was looking at.
What a verdict contains
Each clause assessment returns four things:
| Field | What it is for |
|---|---|
| Verdict | One of Conformant, Partial, Gap or Not addressed |
| Rationale | A plain-language explanation of how the verdict was reached |
| Citations | References to the specific passages the verdict relies on |
| Confidence | How sure the AI is, so you know where to look hardest |
The verdict scale is deliberately small. A numeric score invites argument about the number rather than the requirement; four verdicts map onto the conversation you would actually have with an auditor.
Not addressed is kept separate from Gap on purpose. One means your documents don’t appear to speak to the requirement at all. The other means they speak to it and fall short. Those lead to different work: writing something new, versus fixing something that exists.
Citations work by reference. The AI is given a set of retrieved passages and points to the ones it relied on, rather than paraphrasing your documents back to you. Each citation opens the exact passage in context, so checking a verdict means reading a few paragraphs, not re-reading a policy.
Guidance we use but never cite
A clause on its own is often not enough context to assess it well. Requirements are terse, and the difference between conformant and partial frequently depends on interpretation that experienced auditors carry in their heads.
To help with that, each clause has interpretation notes written by our own staff: what the requirement is getting at, what evidence typically satisfies it, and common ways organisations fall short. The AI reads these notes as context when it assesses the clause.
It never cites them. Citations only ever point at your documents. That line matters. Our guidance is an opinion about what a requirement means; your documents are the evidence of what you do. If the two were mixed in the citations, a verdict could appear to be supported by text that is not yours, and an auditor would rightly reject it.
AI proposes, a person decides
Every finding starts unreviewed. A person then either confirms it, accepting the AI’s verdict, or overrides it with a corrected verdict and a note explaining why.
The original AI verdict is never changed. An overridden finding keeps both the AI’s view and the human decision, and both are recorded in the audit trail. We did this for two reasons:
- Accountability. The management system belongs to you. A verdict that counts should have a named person behind it.
- Honesty about disagreement. If the AI said Gap and you said Conformant, that is worth being able to see later, whether you were right or not. Erasing the AI’s view would make the record tidier and less useful.
Gaps and partials that survive review feed the gap register, where you decide how to close them: write a procedure, raise a CAPA, or link evidence the AI missed.
Evidence stays proposed until confirmed
The same principle applies to evidence. Passages the AI cited become proposed evidence for that clause. They stay proposed until a person confirms or rejects them.
Only confirmed evidence appears in audit-day mode and in the audit pack PDF. Nothing reaches an auditor because the AI thought it was relevant. This adds a step. We think the step is the point: evidence you present at an audit should be evidence you have looked at.
What this doesn’t do
It is worth being clear about the limits.
- It assesses documents, not practice. A conformant verdict means your documents address the requirement. It does not mean the process runs as written. That is what internal audits and records are for.
- Retrieval can miss things. If the relevant passage is phrased unusually or buried in an unrelated document, it may not be retrieved, and the verdict may be harsher than it should be. That is one reason human review exists, and why you can link evidence manually.
- Confidence is a guide, not a guarantee. Low-confidence verdicts deserve a closer look, but high-confidence ones still get reviewed.
In practice this means StayEvident’s outputs look more like an audit workpaper than a consultant’s narrative summary. Some users find this style dry. We think that is the correct trade-off for a tool meant to help organisations prepare for, and get through, external scrutiny. The goal is not to make compliance feel effortless. It is to make your evidence trail robust enough that the audit conversation becomes straightforward.
More detail is in the docs on AI assessments and evidence linking.