AI agents are easy to demo. Production agents are different.

Agent demos are easy. You pick the task, you pick the input, and you run it until it goes well. That is not dishonest — it is how demos work for every kind of software — but it does mean the demo tells you almost nothing about whether the thing will survive contact with an organisation.

Here is what I have found actually separates the two.

An agent inherits your permissions model#

This is the big one, and it is the least discussed.

The moment you give an agent tools, it is operating with some identity, and that identity has some set of permissions. If those permissions are broader than they should be — and in most organisations they are, because access accretes and nobody audits it — the agent will exercise them. Not maliciously. Just thoroughly, and much faster than a person would.

Every organisation has access permissions that are technically wrong and practically harmless because no human happens to use them. An agent is the thing that finds out.

So the unglamorous work — deciding exactly what the agent may read, what it may write, and under whose authority — is not preparation for building the agent. It is building the agent. The prompt is comparatively trivial.

Retries hide failure and multiply side effects#

A retry loop makes an unreliable component look reliable. That is fine when the operation is a read. It is a serious problem when the operation has a side effect.

An agent that half-completed an action, failed, and retried has now done part of that action twice. If the action was "send an email" or "create a record" or "update a status", the retry is not a recovery — it is a second event.

Which means side-effecting operations need to be idempotent, or need to be gated behind something that knows they already happened. That is ordinary distributed-systems discipline, and it arrives in a lot of agent projects late, usually after something duplicated.

You cannot debug what you did not record#

When an agent does something surprising, the question is always: what did it see, what did it decide, and which tools did it call in what order?

If that is not recorded, you are reconstructing behaviour from an outcome, which is close to impossible. Logging the reasoning trace and the tool calls feels like overhead right up until the first time you need it, at which point it is the only thing that matters.

Knowing when to stop#

A model's default is to produce something. Left alone, an agent will keep going, take another action, try another approach — which is exactly what you want in a demo and often wrong in production.

Useful agents know their limits: when to stop, when to ask, when to hand back to a person. In a high-stakes domain that behaviour is the product. When we built the legal AI platform, the design rule was no source, no claim — and most of the engineering went into making the system decline confidently rather than answer fluently.

Confident refusal is harder to build than confident answering. It is also worth more.

Human review is a design decision, not a fallback#

"A human checks it" is only meaningful if the output is checkable. A wall of plausible prose is not — nobody verifies that under time pressure, they skim it and approve.

If review matters, the output has to be structured for verification: discrete items, each traceable to its source. That is a different output format, decided early, not a review step added at the end.

What I actually check before calling something production-ready#

• What can this agent do that it should not be able to do?
• What happens when a tool call fails halfway?
• If it does something wrong next Tuesday, can I reconstruct why?
• What does it do when it does not know?
• Is the output shaped so a person can actually verify it, or only read it?

None of those questions are about the model. That is the point.