OPINION
"No source, no claim" should be the default for high-stakes AI
In legal, compliance and finance, a confident wrong answer costs more than a refusal — and unlike a refusal, it does not announce itself.
There is an asymmetry at the centre of high-stakes AI that most products get backwards.
A refusal costs the user a few minutes. They go and look it up. Mildly annoying, entirely recoverable.
A confident, well-written, wrong answer about a filing requirement costs considerably more. And — this is the part that matters — it does not announce itself. Nobody discovers it at the moment of use. They discover it later, from a regulator, a counterparty, or a colleague who checked.
Once you take that asymmetry seriously, a lot of design decisions that look excessively conservative become obviously correct.
Fluency is a bad proxy and we are all fooled by it
We judge confidence by how something is written. A well-structured answer with appropriate hedging and the right register reads as more reliable than a tentative one.
Language models are extremely good at that register. It is essentially what they optimise for. Which means the surface signal we instinctively use to assess reliability has been decoupled from actual reliability — and we have not adjusted, because the instinct is not something you can decide to switch off.
The only real defence is to stop asking users to judge. Show them where the claim came from and let them check.
Where this actually applies
Legal is the obvious one — it is what I work on — but the same asymmetry holds anywhere the cost of a silent error is high and the error is discovered late:
- Compliance. A wrong answer about an obligation surfaces during an audit.
- Finance. A confidently wrong figure propagates into decisions.
- Medical. Obviously, and with far more serious consequences.
- Enterprise decisions. Wrong analysis that reaches a board paper is very hard to walk back once people have acted on it.
The common feature is not the subject matter. It is the delay between the error and its discovery.
What "no source, no claim" means in practice
Three things, in order of how often they are skipped.
Retrieval gates generation. The system does not answer from parameters. If retrieval does not support an answer, there is no answer.
Citations are carried as data. Not asked for in a prompt. Attached mechanically to the claim they support, so they cannot be plausibly wrong.
Refusal is a designed path. With its own output, ideally explaining what the system does have. "I cannot support that from my sources, but here is what I do have on the adjacent question" is genuinely useful. Silence is not.
The objection, and my answer to it
The objection is that this makes the product worse. Users want answers, a system that frequently declines feels unhelpful, and a competitor that always answers will demo better.
That is true, and it is the wrong frame. In a professional context you are not competing on how often you answer — you are competing on whether a professional can put their name on the output. A tool that is right 95% of the time with no way to tell which 5% is not 95% useful. It is useful only to someone who checks everything, at which point the time saving is gone.
A tool that answers less often and shows its work is one a professional can actually build a workflow around.
The short version
If a system cannot show where a claim came from, a professional cannot use it — independently of whether that claim happens to be correct. Verifiability is not a feature of the answer. It is the thing that makes the answer usable at all.