AI agents that do real work, not demos
What separates an AI agent that survives contact with a real business from an impressive demo, and the guardrails that make one safe to deploy.
An AI agent is a system that takes multiple steps toward a goal without being prompted at each one. It decides what to do next, uses tools, and keeps going until the task is done or it gets stuck.
Demos of this are extraordinary. Production deployments are frequently disappointing. The gap between the two is not model quality. It is engineering.
Why demos succeed and deployments fail
A demo runs once, on a chosen example, with someone watching. Production runs hundreds of times on inputs nobody anticipated, with nobody watching.
The specific things that break:
Compounding error. An agent taking ten steps at 95 percent reliability per step succeeds about 60 percent of the time. At twenty steps it is under 40 percent. Reliability that sounds excellent per step is unusable end to end.
No stopping condition. An agent that cannot recognise it is stuck will retry, rephrase, and burn tokens indefinitely. Every agent needs a step budget and a way to give up honestly.
Silent wrongness. A crashed process is obvious. An agent that confidently files the wrong data is not, and may not be noticed for weeks.
Changing ground. The API it depends on changes a field name and the agent does something subtly wrong rather than failing loudly.
What makes an agent deployable
Fewer steps. The single biggest reliability lever. If a task can be done in three steps rather than twelve, the maths improves dramatically. Much of building good agents is removing the need for the agent to decide things.
Deterministic where possible. If a step can be plain code, make it plain code. Use the model for the parts that genuinely need judgement, and use code for the parts that do not. An agent that calls a well-tested function is far more reliable than one reasoning its way through the same logic.
Hard limits. A step budget, a time budget, and a token budget. All three, enforced by the harness rather than requested in the prompt.
Approval gates on consequences. Reading is cheap to get wrong. Sending, deleting, paying and publishing are not. Anything with an outward-facing or irreversible effect should pause for a human, at least until you have evidence it is safe.
Observability. Log every tool call, every input, every output. When someone asks “why did it do that,” you need an answer. This is not optional in any business context.
Evaluation. A set of cases with known-correct outcomes, run whenever anything changes. Without it you are guessing whether a prompt change helped or hurt, and prompt changes routinely do both at once.
Where agents genuinely pay off
Triage and routing. Read the incoming thing, classify it, route it. Errors are cheap because a misrouted item gets forwarded.
Research and summarisation across sources. Pull from several systems and assemble a summary. The human reviews the result, so mistakes surface.
Structured extraction at volume. Invoices, applications, forms. Genuinely hard with rules, genuinely suited to models.
First-pass drafting. Draft the reply, the description, the report. A person edits before it goes out.
Notice the pattern: a human sees the output before it matters. That is the common thread in agent deployments that work.
Where they usually should not go
Anything requiring exact arithmetic. Anything where a wrong answer is a regulatory or safety problem. Anything replacing a process nobody has written down, because the agent will automate the confusion.
And anything where the honest answer is a scheduled job. A great deal of what gets proposed as agent work is a cron job with extra steps and a much larger bill.
The honest cost picture
Agents cost more to run than single requests, because they make many calls per task. The two things that inflate cost most are unnecessary steps and using the largest model for steps a smaller one handles fine.
A well-scoped agent doing genuine work for a small business is usually tens of dollars a month, not thousands. If a quote is dramatically higher, ask what is driving the step count.
Start with the boring version
Before building an agent, build the thing without one. Write the script that does the task in a fixed order. Frequently that is enough, and it is cheaper, faster and more reliable.
If the fixed order genuinely cannot handle the variety of real inputs, that is your evidence an agent is warranted, and you now have a baseline to measure it against.
If you want a straight assessment of whether your problem needs an agent, a simple automation, or nothing at all, describe what you are trying to do.
Related: what MCP actually is and what to automate first.
Further reading
Topics in this article
Get new articles by email
One email when something new goes up, roughly twice a month. Plain writing on what software costs and what is worth building. No sequences, no sales calls, and one click to leave.
Keep reading on the same subject
-
The guardrails an AI agent actually needs
Agents that take real actions need limits, not enthusiasm. The seven controls worth building, why each exists, and which ones you can skip.
Review the guardrails -
What building an MCP server for your business involves
A practical walkthrough of scoping, building and running an MCP server: which tools to expose, what to withhold, and what it costs each month.
See what a build involves -
What MCP actually is, and why your business should care
A plain explanation of the Model Context Protocol, what it lets AI assistants do with your own systems, and when building an MCP server is worth it.
Understand MCP