AI-written specs sound sourced but often aren't. The Claim Trace is a 15-minute audit that labels every claim Sourced, Inferred, Assumed or Invented before engineering or a coding agent builds from it.
AI hallucination in specs is when an AI-written PRD or spec states something about your users, market or product that no source supports, in the same confident voice as the parts that are true. The fix is a short audit before anyone builds: trace every claim and label it Sourced, Inferred, Assumed or Invented. We call that audit the Claim Trace, and it takes about 15 minutes.
Key takeaways
- NIST lists "confabulation" as one of 12 risks unique to or made worse by generative AI, and warns that models can also produce made-up citations that justify a wrong answer (NIST, 2024).
- Models guess because training and benchmarks reward guessing: 9 of the 10 popular benchmarks one 2025 paper reviewed give zero credit for "I don't know" (Kalai et al., 2025).
- In specs, hallucination rarely invents a feature. It invents a reason: a usage pattern, a churn driver, a "most-requested" ranking.
- Only 3% of developers say they highly trust the accuracy of AI output, and 46% actively distrust it (Stack Overflow, 2025). Specs deserve the same scepticism as code.
- The most useful output of the audit is not the cleaned spec. It is the assumption ledger, which becomes your discovery plan.
What is spec hallucination?
Spec hallucination is a specific kind of what NIST calls confabulation: "confidently stated but erroneous or false content" that can mislead the reader (NIST AI 600-1). In a spec, it lives in the Problem, Context and Success Metrics sections, the parts that are supposed to come from evidence.
It rarely looks like a made-up feature. The feature is plausible and the structure is clean. The invented part is the justification:
- "Admins currently export data manually every week."
- "Reporting was the most-requested capability among churned accounts."
- "We expect a meaningful lift in activation."
Each reads like a finding. None may be one. The model has read thousands of PRDs, so it produces the shape of evidence: a behaviour, a segment, a direction of change. If your inputs were thin, it fills the gaps with what is statistically typical for a spec like yours.
Intrinsic vs extrinsic hallucination in a spec
Researchers split hallucinations into two types (Ji et al., 2022), and both show up in specs:
| Type | Definition | What it looks like in a spec |
|---|---|---|
| Intrinsic | The output contradicts the source you gave the model | Your interview notes say 3 of 12 customers mentioned exports; the spec says "most customers" |
| Extrinsic | The output can be neither supported nor contradicted by the source | The spec says "admins spend hours in spreadsheets" and nothing you supplied mentions time at all |
Extrinsic hallucinations are the harder ones to catch, because there is nothing to compare them against. The reader has to notice that a sentence has no source, which is exactly what fluent writing hides.
Why do AI models invent claims in specs?
A 2025 paper by researchers at OpenAI and Georgia Tech argues that models hallucinate because training and evaluation reward guessing over admitting uncertainty, much like a student guessing on a multiple-choice exam (Kalai et al., 2025). Of 10 widely used benchmarks the authors reviewed, 9 grade answers strictly right or wrong and give no credit for "I don't know". Under that scoring, a confident guess always beats an honest blank.
The paper also explains why some facts are more exposed than others. If 20% of a type of fact (their example is people's birthdays) appears only once in the training data, you should expect a base model to hallucinate on at least 20% of those facts. When the authors asked one large open model for a birthday and said "only answer if you know", it gave three different wrong dates.
Now apply that to your spec. Facts about your customers, your churned accounts and your admins' weekly habits appear zero times in any training data. The only way a model can state them is if you supplied them. Anything specific about your users that you did not provide is, by construction, a guess dressed as a finding.
NIST adds one more warning that matters for product documents: generated text may include confabulated logic or citations that justify the answer, which makes readers more likely to trust it (NIST, 2024). A spec that says "as customers told us in Q3" is more persuasive than one that just asserts, whether or not the Q3 conversation exists.
Why is AI hallucination in specs more expensive in 2026?
Three things changed at once.
Drafting cost collapsed. A spec used to take a PM a day, and that day forced contact with the evidence. Now the draft arrives in under a minute, before anyone has reopened the research.
Downstream cost did not collapse. If an engineer or an AI coding agent builds from the spec, a fabricated premise becomes working code within hours. You find out the premise was invented after the feature ships, not before.
Review habits didn't adapt. Teams review specs for clarity, completeness and tone, which are the properties language models are best at producing. Almost nobody reviews for provenance. Developers have already learned this lesson with code: 66% name "AI solutions that are almost right, but not quite" as their biggest frustration, and 45% say debugging AI-generated code takes longer (Stack Overflow, 2025). An almost-right spec is worse, because nothing fails to compile.
Professional documents are not immune. In October 2025, Deloitte agreed to repay the final instalment of an AU$440,000 report for an Australian government department after a researcher found fabricated academic references and a made-up quote from a court judgment. The corrected version removed more than a dozen bogus references and footnotes and disclosed that a GPT-4o based tool chain had been used (The Register, 2025). A spec rarely gets that much outside scrutiny, which is why the check has to happen inside the team.
The fix is not "don't use AI to draft". It is to add the review step the draft skipped: trace every claim to a source.
How do you audit an AI-written spec? The Claim Trace
Read the spec once, sentence by sentence. Every sentence that asserts something about users, the market, the product or the future gets exactly one of four labels.
| Label | Definition | The test | What you do with it |
|---|---|---|---|
| Sourced | Points to something you can open: an interview, a ticket, a query result, a signed decision | Can a teammate who wasn't there click through and see the same thing? | Attach the link inline |
| Inferred | A reasonable conclusion from sourced material, but the conclusion itself is new | Can you name the sources it was inferred from? | Attach the sources and make the inference visible ("Based on the 3 calls linked, we think…") |
| Assumed | Something the team believes but has not checked | Would anyone on the team bet money on it without checking? | Move it to the assumption ledger with an owner and a test |
| Invented | No source, no inference chain, nobody can say where it came from | Did a human or a source put this here, or did the model? | Delete it, or turn it into a question |
Sourced
"Three of the last five onboarding calls mention exporting to spreadsheets" is sourced if you can link those three calls. The test is mechanical: a teammate who wasn't in the room can open the link and see the same thing.
Inferred
"Admins likely export because they report upward" is inferred from those calls. That is fine, as long as it is labelled and the underlying sources are attached. Inference is where product judgement lives; the problem is only unlabelled inference.
Assumed
Assumptions are not shameful. Specs run on them. The failure is when an assumption is written in the same voice as a finding, so the reader absorbs it instead of questioning it.
Invented
This is the label that matters most. Invented claims are the model's contribution to the document, and they are the ones to delete or convert into questions.
Do not argue about edge cases between labels. If you hesitate between Inferred and Assumed, pick Assumed. The label is a prompt to look closer, not a verdict.
How do you run the Claim Trace in 15 minutes?
- Minutes 0-3: extract. Copy every factual claim into a scratch list. Skip pure requirements ("the export must be a CSV"); those are decisions, not claims. Look for sentences that start with "users", "customers", "admins", "currently", "most", "because", or a number.
- Minutes 3-8: trace. For each claim, try to find the source in under 60 seconds. If you can't, it is not Sourced. Don't go hunting for a justification after the fact: searching until something loosely fits is how invented claims get laundered into sourced ones.
- Minutes 8-12: act on each label. Sourced gets a link. Inferred gets links plus visible reasoning. Assumed moves to the ledger with an owner and a cheapest test. Invented is deleted or rewritten as a question.
- Minutes 12-15: re-read the spec. If the Problem section collapsed when the invented claims came out, you have learned the most valuable thing the audit can tell you: the feature does not have a problem yet.
A simple decision rule for the end of step 4: if more than a third of the claims in the Problem section came back Invented, send the spec back to discovery rather than to engineering.
What does a Claim Trace look like on a real draft?
Take a made-up but typical AI draft for an admin export feature. The Problem section says:
Admins export data manually every week, spending hours in spreadsheets. Reporting is the top request among churned accounts. A native export will reduce churn.
Run the Claim Trace:
| Claim | Label | Action |
|---|---|---|
| Admins export data manually every week | Assumed (one call mentioned it, nothing about frequency) | Move to the ledger; test with a usage query |
| Spending hours in spreadsheets | Invented | Delete |
| Reporting is the top request among churned accounts | Invented (no ranking exists anywhere) | Delete; replace with a question |
| A native export will reduce churn | Assumed | Keep as a hypothesis with a metric and a date |
Four claims: two deleted, two demoted, zero sourced. The rewritten Problem section is shorter and honest:
One onboarding call (linked) mentioned exporting account data to a spreadsheet. We don't yet know how often admins export or why. Open question: what did churned accounts say when they left? Hypothesis: a native export reduces churn among accounts with more than 5 admins. We will check this before committing to a build.
That document can still lead to a build. But the team now knows what it is betting on, and the person reading it can disagree with the assumption instead of absorbing it.
What goes in the assumption ledger?
The ledger is the list of every Assumed claim, and it is your discovery plan. Record four fields per row:
ASSUMPTION LEDGER: [spec name] | # | Assumption (as written in the spec) | Owner | Cheapest test | Decide by | |---|---|---|---|---| | 1 | Admins export account data weekly | [name] | Query export-button clicks over the last 30 days | [date] | | 2 | A native export will reduce churn | [name] | Ask 5 churned accounts why they left, before building | [date] | | 3 | | | | | A spec with a short, owned ledger can move fast safely. A spec with no ledger either has no assumptions, which is almost never true, or has hidden them. For a fuller method of ranking assumptions by importance and evidence, see our guide to assumption mapping, or sort them on the free assumption map.
How do you reduce AI hallucination in specs before they are written?
A manual trace works, but it relies on discipline at the exact moment discipline is cheapest to skip. It is better to ask for provenance while the draft is produced. Anthropic's documentation lists the core techniques (Anthropic):
- Permit "I don't know". Explicitly allow the model to say it lacks information.
- Quote first. For long inputs (Anthropic gives over 20k tokens as the threshold), ask the model to extract word-for-word quotes before it writes anything.
- Cite, then retract. Ask the model to find a supporting quote for each claim after drafting, and remove any claim it cannot support.
- Restrict knowledge. Tell it to use only the documents you supplied, not its general knowledge.
The Kalai paper suggests a further idea: state a confidence target in the prompt. For example, answer only if you are more than 75% confident, because a wrong answer costs 2 points and "I don't know" costs nothing (Kalai et al., 2025). Applied to specs, that means rewarding the model for writing "no source" instead of filling the gap fluently.
Here is a prompt block you can paste under any spec request:
Use ONLY the research notes, tickets and data I have attached. For every sentence about users, customers, the market or expected results: - add [S: source] if a specific attached item supports it, - add [I: sources] if it is your inference from attached items, - add [A] if it is an assumption with no source. Do not state numbers, rankings or frequencies that are not in the attachments. If a section has no supporting material, write "No evidence yet" and list the questions we would need to answer. After drafting, check each [S] claim against the attachment and remove any claim you cannot support with a direct quote. Anthropic is clear that these techniques reduce hallucinations but do not eliminate them. The prompt makes the Claim Trace faster; it does not replace it. If you are drafting the whole PRD with AI, our AI PRD guide covers the rest of the workflow.
Common mistakes when checking AI-written specs
- Reviewing for polish instead of provenance. Clarity and tone are the things models do best. Ask "where did this come from?" before "is this well written?"
- Hunting for a source after the fact. If a claim needs five minutes of searching to justify, it was Assumed or Invented when it was written.
- Treating requirements as claims. "Export must support CSV" is a decision. Don't waste the 15 minutes tracing decisions.
- Auditing only the Problem section. Success metrics hallucinate too. "A 15% lift in activation" with no baseline is Invented, however precise it looks.
- Letting the ledger rot. An assumption with no owner and no date is a wish. If the ledger isn't reviewed before the build starts, the audit was theatre.
Where does the Claim Trace stop?
The method tells you whether a claim has a source. It does not tell you whether the source is good. A fully sourced spec can still describe the wrong thing, because sources can be biased, thin or unrepresentative. Pair the trace with a check on evidence quality: how many independent sources, how recent, and how close to a real buying or usage decision. Our post on what counts as evidence for each product risk gives a quick scale for this.
It also covers only part of the risk. Marty Cagan's four big risks are value, usability, feasibility and business viability, and he puts value and viability on the product manager (SVPG, 2017). AI has made a working prototype cheap, which shrinks some feasibility questions. It has not made value or viability any easier to know, and those are exactly the sections where invented claims cluster.
Finally, the audit does not replace judgement about what to build. It makes judgement visible. Once a spec has passed, hand it over in a format engineers will use: our guide to the technical spec engineers actually read covers that step, and the acceptance checklist for AI-generated code covers reviewing what comes back.
Where Specky fits
The Claim Trace is easiest when provenance is captured at write time. That is the design stance behind Specky. Interviews, tickets, feedback and decisions live as connected nodes in a Product Graph, and when the AI drafts a spec or insight it cites the specific evidence it used, inline. Drafts land in the PM Inbox for human review rather than going straight into the record, so a reviewer can see which sentences carry citations and which carry none. You can approximate the same thing in any tool with the prompt above; the point is that "no source" should be visible, not hidden.
Frequently asked questions
What is AI hallucination in specs?
It is when an AI-written PRD or spec states something about users, the market or expected results that no source supports, written in the same confident voice as the sourced parts. It usually appears as an invented reason (a usage pattern, a ranking, a churn driver) rather than an invented feature, which is why reviewers miss it.
How long does a Claim Trace take?
About 15 minutes for a typical one-to-three-page spec: 3 minutes to extract claims, 5 to trace them, 4 to act on each label and 3 to re-read. Give yourself 60 seconds per claim to find a source. Longer specs scale roughly with the number of factual claims, not the page count.
Can I ask the AI to check its own spec for hallucinations?
Partly. Asking the model to find a supporting quote for each claim and retract any it can't support is one of Anthropic's recommended techniques, and it catches many extrinsic hallucinations. It is not enough on its own, because the model can misread a source or quote something that doesn't actually support the claim. A human still checks the links.
What is the difference between an assumption and a hallucination?
An assumption is a belief the team holds and knows it hasn't checked. A hallucination is a claim with no origin that nobody on the team actually believed until the model wrote it. Labelled assumptions are useful because they become tests. Hallucinations should be deleted or turned into questions.
Should we stop using AI to write PRDs?
No. AI drafting saves real time on structure, edge cases and wording. The risk sits in the evidence sections. Supply the research, tickets and data as inputs, ask the model to mark sources and assumptions, and run a Claim Trace before the spec goes to engineering or to a coding agent.
What should I do if most of the Problem section is invented?
Stop the build and go back to discovery. If more than a third of the problem claims are invented, the feature doesn't yet have a demonstrated problem. Turn each invented claim into a question, pick the two that would most change the decision, and test those first with interviews, a usage query or a support-ticket review.
Sources
- NIST — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1), July 2024
- Kalai, Nachum, Vempala and Zhang — Why Language Models Hallucinate (arXiv, September 2025)
- Ji et al. — Survey of Hallucination in Natural Language Generation (arXiv, 2022)
- Anthropic — Reduce hallucinations (Claude documentation)
- Stack Overflow — 2025 Developer Survey: AI
- The Register — Deloitte refunds Aussie gov after AI fabrications slip into $440K welfare report (October 2025)
- SVPG (Marty Cagan) — The Four Big Risks (2017)
This story was originally published on 19 October 2026 at https://www.specky.space/blog/spec-hallucination-claim-trace-ai-written-specs.

