Skip to content
AI Agents & MCP

AI agents that do real work, not demos

What separates an AI agent that survives contact with a real business from an impressive demo, and the guardrails that make one safe to deploy.

4 min read
AI agents that do real work, not demos. An article by Athabasca Solutions.

An AI agent is a system that takes multiple steps toward a goal without being prompted at each one. It decides what to do next, uses tools, and keeps going until the task is done or it gets stuck.

Demos of this are extraordinary. Production deployments are frequently disappointing. The gap between the two is not model quality. It is engineering.

Why demos succeed and deployments fail

A demo runs once, on a chosen example, with someone watching. Production runs hundreds of times on inputs nobody anticipated, with nobody watching.

The specific things that break:

Compounding error. An agent taking ten steps at 95 percent reliability per step succeeds about 60 percent of the time. At twenty steps it is under 40 percent. Reliability that sounds excellent per step is unusable end to end.

No stopping condition. An agent that cannot recognise it is stuck will retry, rephrase, and burn tokens indefinitely. Every agent needs a step budget and a way to give up honestly.

Silent wrongness. A crashed process is obvious. An agent that confidently files the wrong data is not, and may not be noticed for weeks.

Changing ground. The API it depends on changes a field name and the agent does something subtly wrong rather than failing loudly.

What makes an agent deployable

Fewer steps. The single biggest reliability lever. If a task can be done in three steps rather than twelve, the maths improves dramatically. Much of building good agents is removing the need for the agent to decide things.

Deterministic where possible. If a step can be plain code, make it plain code. Use the model for the parts that genuinely need judgement, and use code for the parts that do not. An agent that calls a well-tested function is far more reliable than one reasoning its way through the same logic.

Hard limits. A step budget, a time budget, and a token budget. All three, enforced by the harness rather than requested in the prompt.

Approval gates on consequences. Reading is cheap to get wrong. Sending, deleting, paying and publishing are not. Anything with an outward-facing or irreversible effect should pause for a human, at least until you have evidence it is safe.

Observability. Log every tool call, every input, every output. When someone asks “why did it do that,” you need an answer. This is not optional in any business context.

Evaluation. A set of cases with known-correct outcomes, run whenever anything changes. Without it you are guessing whether a prompt change helped or hurt, and prompt changes routinely do both at once.

Where agents genuinely pay off

Triage and routing. Read the incoming thing, classify it, route it. Errors are cheap because a misrouted item gets forwarded.

Research and summarisation across sources. Pull from several systems and assemble a summary. The human reviews the result, so mistakes surface.

Structured extraction at volume. Invoices, applications, forms. Genuinely hard with rules, genuinely suited to models.

First-pass drafting. Draft the reply, the description, the report. A person edits before it goes out.

Notice the pattern: a human sees the output before it matters. That is the common thread in agent deployments that work.

Where they usually should not go

Anything requiring exact arithmetic. Anything where a wrong answer is a regulatory or safety problem. Anything replacing a process nobody has written down, because the agent will automate the confusion.

And anything where the honest answer is a scheduled job. A great deal of what gets proposed as agent work is a cron job with extra steps and a much larger bill.

The honest cost picture

Agents cost more to run than single requests, because they make many calls per task. The two things that inflate cost most are unnecessary steps and using the largest model for steps a smaller one handles fine.

A well-scoped agent doing genuine work for a small business is usually tens of dollars a month, not thousands. If a quote is dramatically higher, ask what is driving the step count.

Start with the boring version

Before building an agent, build the thing without one. Write the script that does the task in a fixed order. Frequently that is enough, and it is cheaper, faster and more reliable.

If the fixed order genuinely cannot handle the variety of real inputs, that is your evidence an agent is warranted, and you now have a baseline to measure it against.

If you want a straight assessment of whether your problem needs an agent, a simple automation, or nothing at all, describe what you are trying to do.

Related: what MCP actually is and what to automate first.

Further reading

Sections covered in AI agents that do real work, not demos: Why demos succeed and deployments fail, What makes an agent deployable, Where agents genuinely pay off, Where they usually should not go, The honest cost picture, Start with the boring version
The shape of the argument, in order.

Get new articles by email

One email when something new goes up, roughly twice a month. Plain writing on what software costs and what is worth building. No sequences, no sales calls, and one click to leave.

We use it for the newsletter and nothing else. Unsubscribe any time.

Have something you need built?

Tell us what the problem is. You will get an honest read on whether it is worth building, what it would take, and roughly what it would cost. No pitch deck, no pressure.

Replies within one business day. Mon to Fri, 9am to 5pm MT.

Call Start a project