What actually makes something an agent
Pull-quote: “Count the model calls across a hundred runs. If the number never varies, your code chose the sequence, and what you built is a workflow.”
An agent is a system in which a model chooses the next action at run time, that action is executed against a real system, the result is returned to the model, and the cycle repeats until a stop condition. Three properties, all of them required. Remove the run-time choice and you have a workflow. Remove the execution and you have a plan. Remove the feedback and you have a sequence of independent calls.
That definition exists so a team can finish an argument. Two engineers who both say “agent” and mean different rungs of capability will disagree about scope, budget, and risk for weeks without ever locating the disagreement. This post gives you the definition, a five-rung ladder from scripted workflow to multi-agent system, and four tests that place any existing system on the ladder in about ten minutes.
It is written for whoever has to write the design document. No framework knowledge is assumed, and the tests work on a system you did not build, using only its logs.
What you will be able to do
- State a one-sentence definition of an agent that distinguishes it from a workflow, a chatbot, and a pipeline.
- Place any system on a five-rung ladder: scripted workflow, tool-calling assistant, single-loop agent, multi-step planner, multi-agent system.
- Run four tests against a running system’s logs to find its actual rung, not its claimed one.
- Name the components each rung obliges you to build, and the specific failure that follows from skipping them.
- Recognise the signals that look like agency and are not, including a chat interface, an LLM, MCP, and retrieval.
- Argue for staying on a lower rung with a reason stronger than caution.
What is an agent?
An agent is a system in which a model chooses the next action at run time, executes it against the world, observes the result, and repeats until a stop condition. The three clauses each rule something out.
Run-time choice. The sequence of steps is not written in your code before the run begins. If a function calls the model, then always calls a search tool, then always calls the model again, the sequence was chosen by a programmer. This is the distinction Anthropic draws when it separates the two: workflows are systems where models and tools are orchestrated through predefined code paths, and agents are systems where models dynamically direct their own processes and tool usage (Building effective agents).
Execution against the world. The chosen action produces an effect or retrieves state outside the model. A model that writes a numbered plan and stops has not acted; it has produced text describing action. This matters because everything difficult about agents, permissions, verification, idempotency, and blast radius, arrives with the first real side effect.
Feedback that changes the next choice. The result of the action is returned to the model and influences what it does next. Without this you have a fan-out of independent calls, which is a useful pattern and not an agent, because nothing accumulates.
Two supporting terms, defined once and used consistently for the rest of this post. A step is one iteration: a model call, the tool executions it requested, and the observation that follows. A stop condition is a check the system makes on itself to end a run, whether that is the goal being met, a budget being spent, or an escalation to a human. Every agent needs an explicit stop condition, because run-time choice of the next action includes the possibility of choosing wrongly forever.
What are the five rungs of the agent capability ladder?
Five rungs, ordered by how much of the control flow the model decides rather than your code.
Rung one, scripted workflow. Your code decides the sequence of steps, and the model fills in content at fixed points. Prompt chaining, classification followed by a branch, extract then validate then write. The number of model calls is a property of the code path, and you can draw the whole graph before the run. Rung one is the correct answer for most business processes, and it is also where most systems described as agents actually sit.
Rung two, tool-calling assistant. The model picks a tool and arguments, one hop, and control returns to the caller with the result. The model chose what, your code chose when, and there is no second iteration unless a human sends another message. A retrieval-augmented question answering system with function calling lives here. So does most of what ships as an “AI assistant”.
Rung three, single-loop agent. The model chooses the next action repeatedly, against observations, until a stop condition. This is the first rung that meets the definition. The loop is short to write and the difference it makes is total: the number of steps is now decided at run time by what the tools returned, which means the run has no natural end and you have to supply one.
# Rung two: one hop, the caller decides there is no next step.
reply = model(messages, tools=schemas)
if reply.calls:
result = dispatch(reply.calls[0])
return summarise(result)
# Rung three: the model decides whether there is a next step, you decide the ceiling.
for step in range(1, MAX_STEPS + 1):
reply = model(messages, tools=schemas)
if not reply.calls:
return finish("model_stopped", reply)
messages += [reply] + [tool_result(c, dispatch(c)) for c in reply.calls]
return finish("budget_exhausted", None)Rung four, multi-step planner. The system holds an explicit representation of intent that outlives a single step and gets revised when observations contradict it. The distinguishing artifact is the plan itself: a list of tasks, a graph, a set of open subgoals, something you could print. A rung-three loop reconsiders everything from the message history each step. A rung-four planner compares what it observed against what it intended, and replans when those diverge. Orchestrator-workers, where a central model breaks a task into subtasks it could not have predicted and synthesises the results, is the common shape.
Rung five, multi-agent system. Two or more loops run with separate context windows and a defined handoff contract between them. The requirement is both halves. Several prompts inside one context window are one agent wearing hats. Separate contexts without a contract are two agents that will drift, duplicate work, and disagree about whether a task is done.
How do you tell which rung you are actually on?
Four tests, applied in order, using the system’s logs rather than its architecture diagram. Stop at the first one that fails.
Test one, the call count. Take a hundred completed runs and count model calls per run. If every run made the same number, your code chose the sequence and you are on rung one, whatever the repository is called. If the count varies with the input but the variation maps exactly to which branch the code took, you are still on rung one. You need the count to vary because of what a tool returned.
Test two, the second iteration. Find a run where the model called a tool, saw the result, and then called a different tool it had not called before. If no run in your sample does this, you are on rung two: the model chose a tool but never chose again in light of an observation. This test is more reliable than reading code, because a loop that exists but always exits after one iteration is a rung-two system with extra machinery.
Test three, the surviving artifact. Ask whether there is an object representing intent that persists across steps and gets modified. Not the message history, which accumulates by default and represents what happened rather than what is meant to happen. A task list the system rewrites, a set of subgoals it marks complete, a plan it revises. If there is one, you are on rung four. If the only durable object is the transcript, you are on rung three.
Test four, the contract. Ask whether two loops have separate context windows and a written handoff contract: what gets passed, in what shape, who owns the outcome, and what happens when the receiver rejects it. Both conditions, or you do not have a multi-agent system. Two loops with a shared window are one agent. Two loops with separate windows and no contract are a distributed system without a protocol, which is the most expensive thing on this ladder to debug.
The tests are ordered because the properties are cumulative. A system cannot be a multi-step planner without being a loop first, and a claim to rung five from something that fails test two is the most common misdescription in this space.
What does not make something an agent?
Six things that look decisive and are not. Each of these is compatible with every rung, including rung one.
| Signal | Why it does not decide the rung |
|---|---|
| A chat interface | The interface is where a human types. Rung one systems have chat windows and rung five systems run on a cron schedule. |
| Using an LLM | Every rung uses one. The question is who chooses the next step, not what generates the text. |
| Calling tools through MCP | The Model Context Protocol standardises how tools are exposed. It says nothing about who decides when to call them. |
| Retrieval or RAG | Fetching documents before generating is a step. A workflow with retrieval is still a workflow. |
| Multiple prompts or personas | Several roles inside one context window is one loop with variable instructions, not several agents. |
| Autonomy claimed in the interface | “Working on it” and a progress spinner are presentation. Test one still applies. |
The reverse is also worth stating, because it catches teams underestimating what they have. A system with no chat interface, no persona, and no framework can be a rung-three agent if the number of model calls depends on what the tools returned and nothing in the code caps it. Those systems exist in production, usually as a cron job someone extended twice, and they are the ones running without a budget.
What does each rung oblige you to build?
Each rung adds a required component, and skipping it produces a specific, predictable failure rather than a general degradation.
At rung one you owe input validation and per-node error handling, and the failure from skipping it is a malformed intermediate that propagates to the output. Ordinary software engineering, ordinary consequences.
At rung two you owe typed tool schemas and argument validation before execution. Skip it and the model’s plausible-looking argument reaches your system as a real call, which is how a date range of “last quarter” becomes a table scan. This is the subject of designing tools an agent can actually use.
At rung three you owe four things, and this is the largest single jump on the ladder. A step budget enforced in code, not requested in the prompt. A stop condition that produces a named exit reason. A trajectory log of each step’s prompt, call, result, and cost. And verification: an independent read of the source system confirming the intended change actually happened, rather than the model’s report that it did. Skip the budget and runs do not terminate. Skip verification and the agent reports success on work that never landed, which is the most expensive failure class because it is silent. The loop mechanics are covered in designing a loop that closes.
At rung four you owe a plan representation you can inspect, a defined trigger for replanning, and a progress check that can tell the difference between working and thrashing. Skip the progress check and the planner revises the same plan indefinitely, each revision locally reasonable.
At rung five you owe a handoff contract, per-agent context isolation, per-agent budgets, and a termination condition for the system rather than for each loop. Skip system-level termination and two agents can wait on each other while both remain individually healthy, which no per-agent timeout detects. The economics of when this rung pays are in the orchestrator and worker model loop.
When should you stay on a lower rung?
Stay lower whenever the sequence of steps is genuinely knowable in advance, because a workflow you can draw is a workflow you can test exhaustively, and an agent is not. The argument for the lower rung is not caution, it is that predictability is a feature you are giving up in exchange for handling cases you could not enumerate. If you can enumerate them, you are paying and receiving nothing.
Three concrete signals that rung one is correct. The process has a compliance description that names the steps in order. The set of inputs is bounded and you have seen most of them. The cost of a wrong step exceeds the cost of failing to handle an unusual input.
Three signals that rung three earns its cost. You cannot predict how many steps a task needs, because that depends on what the first tool call returns. The space of valid inputs is open, so enumeration is not available. And you have somewhere for the run to be verified, meaning a source of truth the system can read back rather than a human who checks a sample.
Rung five deserves a higher bar than it usually gets. Add a second loop when two parts of the work need genuinely different context, not when they need different instructions. Different instructions is a prompt change. Different context is an architecture change, and it brings the handoff contract, the isolation, and the system-level termination condition along with it.
Where this goes wrong
The system is described one rung above where it sits. The symptom is a design review where the risks discussed do not match the risks present, usually planning failures debated for a system that has never run two iterations. The cause is that “agent” was used as a category rather than a claim. Run test one before the review and put the model-call distribution in the document.
The loop exists but never loops. The symptom is a rung-three architecture with rung-two behaviour, and a team surprised that quality did not improve. The cause is usually a system prompt that pushes toward an immediate answer, or a tool set that returns everything in one call. Check the distribution of iteration counts: if ninety percent of runs exit after one step, you have paid for a loop and not received one.
Multi-agent was adopted to solve a context problem. The symptom is several agents whose outputs contradict each other. The cause is that the original agent was over-filled with context, and splitting it into roles seemed like the fix. It is not; the fix is a context budget. Splitting adds a coordination problem on top of the context problem you still have.
The plan is a prompt instruction. The symptom is a system claimed to be at rung four where nothing is inspectable at run time. The cause is a plan that only exists as text inside a message rather than as an object your code can read, compare, and revise. If your code cannot answer “which subgoals are still open” without asking the model, you are on rung three with an elaborate prompt.
Common questions
What is the difference between an AI agent and a workflow?
A workflow’s sequence of steps is decided by code before the run starts, and an agent’s next step is chosen by the model during the run based on what it just observed. The practical test is whether the number of model calls varies because of what a tool returned. A workflow is more predictable and exhaustively testable; an agent handles cases you could not enumerate in advance.
Is a chatbot with tool calling an agent?
Not on its own. A chatbot that picks a tool, gets a result, and answers has made one choice and returned control to the human, which is rung two on the ladder. It becomes an agent at the point where it decides, without being asked again, that the result of the first tool means a second tool should be called.
Does using MCP make my system agentic?
No. The Model Context Protocol is a standard for exposing tools, data, and prompts to a model, and it is equally usable from a fixed pipeline and from an autonomous loop. It changes how tools are connected, not who decides when to call them. A scripted workflow that reaches its tools over MCP is still a scripted workflow.
How many agents make a multi-agent system?
Two loops with separate context windows and a defined handoff contract. The separate context is what makes them distinct systems rather than one system with different instructions, and the contract is what makes them a system rather than two programs sharing a database. Several personas inside one context window is one agent.
Which rung should most teams build?
Rung one for processes you can draw, and rung three for work where the number of steps depends on what the tools return. Rung four and rung five are answers to specific problems, an unpredictable task decomposition and genuinely separate contexts, and adopting either without that problem adds coordination cost with no capability in return.
Next steps
Run test one this week. Pull a hundred completed runs, plot the model calls per run, and put the histogram in your design document. If it is a single spike, you have a workflow and the roadmap items about planning and autonomy can be closed. If it has a long tail, find the longest run and check whether anything in code would have stopped it.
Then read the agent harness, not the model, decides if your agent works, which specifies the nine components rung three obliges you to build, and what agent failures look like when you name them for the failure modes each rung introduces. When the question turns to where the thing runs, how much of the reasoning loop you hand to AWS and where the loop lives on serverless runtimes both assume the vocabulary above.
We hold most production work at rung three deliberately, and the systems that do run more than one loop, including the geopolitical intelligence work behind Aquil, have a written handoff contract before the second loop exists. What we test in public is in AI Fieldwork.