The agent harness, not the model, decides if your agent works
Pull-quote: “A stronger model does not add the verification step you never wrote. It just produces a more convincing report of work it never did.”
Your agent harness, not the model inside it, decides whether your agent works. When a production agent stalls, loops, reports success on work it never did, or forgets a constraint from eight steps ago, the cause is almost never the model’s reasoning. It is the code around the model call: what got assembled into the prompt, which tools were exposed, whether anything checked the result, and what was supposed to stop the run.
This post opens a ten-part course on building, operating, evaluating, and securing an LLM harness. It is for engineers who have made a model call work and now need a system that survives forty steps, a tool that errors, and a user who changes the goal halfway through. It also sets the vocabulary the other nine posts use.
What you will be able to do
- Define a harness precisely, and name its nine components without hand-waving.
- Sort any agent failure into model-bound or harness-bound, and fix the harness-bound majority first.
- Explain why a model upgrade amplified some problems instead of removing them.
- Write a minimum viable harness in one file, and list what production adds to it.
- Choose between LangGraph, CrewAI, LlamaIndex, a vendor harness, and your own thin loop.
- Keep what carries business value independent of the framework you chose.
What is an agent harness?
An agent harness is the code around the model call: everything that decides what the model sees, what it may do, when it runs again, and when it stops. The model contributes one thing, a token stream in response to a prompt. Nearly every behaviour a user would call agentic is harness behaviour.
Nine terms carry the rest of this course, and later posts assume them.
| Term | Definition |
|---|---|
| Harness | The code around the model call: loop, state, context assembly, tool routing, verification, budgets, persistence, recovery, observability. |
| Loop | Assemble context, call the model, execute what it asked for, observe the result, decide whether to repeat. |
| Step | One iteration of the loop: a model call, its tool executions, and the observation that follows. |
| Tool | A typed function the model requests by name, with declared arguments, side effects, and permission scope. |
| Memory | State that outlives its step. Semantic holds facts, procedural holds how to do a task, episodic holds what happened. |
| Budget | A ceiling in code on steps, tokens, seconds, dollars, or calls to one tool. |
| Stop condition | A check the harness makes itself to end a run: goal met, budget spent, no progress, or escalation. |
| Verification | An independent read of the source system confirming the intended change actually happened. |
| Trajectory | The ordered record of a run: prompts, outputs, tool calls, results, verification, cost. The unit of debugging. |
Two of those do most of the work. A budget lives in code: “use no more than ten steps” in a system prompt is a suggestion, while a counter that raises at ten is a budget. Verification is a fresh read of the world, not a confidence question put to the model. If the only evidence a record changed is the model saying so, the run is unverified however detailed the summary.
What does a production harness contain, component by component?
A production harness contains nine components, and each owns a distinct class of failure. The model sits underneath all of them.
Four components form the runtime path. Loop control sequences steps, decides whether a failure means retry or replan, and enforces that verify follows act. Context assembly builds each step’s prompt from system instructions, tool schemas, memory reads, retrieved evidence, and running history; it owns forgotten constraints and silent truncation, and it is the subject of context engineering as a budget later in this course. Tool routing and execution decides which tools are visible this step, validates arguments, enforces permissions, and normalises errors into something the model can act on. Beneath those sits the model call.
Five components cut across every step. Verification re-reads the source system and compares observed to intended; it owns hallucinated success, the most expensive failure class because it is silent by construction. State and memory hold the plan, scratchpad, and durable facts, and own contradiction and staleness. Budgets and stop conditions cap the run and supply the exit reason. Persistence checkpoints so a restart resumes instead of starting over. Error recovery classifies a failure as transient, semantic, or unrecoverable, then routes it to backoff, replanning, or escalation. Observability writes the trajectory, and owns the failure you cannot reproduce.
The split earns its keep after an incident, when you should be able to name the component at fault in one sentence. A review ending with “the model got confused” has no component boundaries and no action item.
Why does a better model not fix a broken agent?
A better model raises the quality of each step, while the harness decides how many steps happen, what each step sees, whether anything checks the result, and when the run ends. Improving the step does not install a verification stage or a budget. The two compose in one direction only: a stronger model amplifies a good harness, because more of its capability survives into the outcome, and it cannot rescue a bad one, because the failure happens in code the model never influences.
There is a sharper version. In a harness with no verification, a stronger model makes the failure harder to catch, because it writes a coherent account with plausible specifics and hallucinated success passes the review a vaguer summary would have failed. That is how a team reports an upgrade made the agent better while the incident rate stays flat.
Sort your failures before you spend anything on an upgrade. Take the last twenty failed runs and ask one question of each: would this have happened if the model had reasoned perfectly at every step? A constraint dropped at step three survives a perfect model, because it was never in the prompt. A run that never ends survives one too, because nothing was checking. An arithmetic slip does not survive, and neither does a shallow plan on a hard decomposition.
Some failures are mixed. Schema compliance is the clearest: stronger models emit valid JSON more often, and constrained decoding plus a validate-and-repair step makes compliance a property of your code. When both fixes exist, take the harness fix, because it holds across model changes.
What is the minimum viable harness, and what does production add?
A minimum viable harness is about fifty lines: a loop, a step budget, a tool registry with typed arguments, an append of every result to the message list, a stop check, and a trajectory log. Below that you do not have an agent, you have a call site.
MAX_STEPS = 12
def run(goal, tools, log):
messages = [system_prompt(tools), user(goal)]
for step in range(1, MAX_STEPS + 1):
reply = call_model(messages, tools=schemas(tools)) # your provider SDK
log.step(step, messages, reply) # trajectory, every step
if reply.calls:
for call in reply.calls:
result = tools[call.name].invoke(call.args) # validates args, may raise
messages += [reply, tool_result(call, result)]
continue
if goal_satisfied(reply, goal): # a check, not a vibe
return log.finish("goal_satisfied", reply)
messages.append(user("Not done. State the next concrete action."))
return log.finish("budget_exhausted", None)call_model stands in for your provider’s client. Two properties matter more than the specific lines: every run ends with a named reason rather than falling off the end, and every step appends to a trajectory, so a failed run leaves evidence. You know it is wired correctly when a run that hits the ceiling reports run 8817 finished: budget_exhausted after 12 steps.
Production adds seven things, each with a triggering incident behind it.
| Production addition | The incident that forces it |
|---|---|
| Verification as its own step | An agent reported an update that never landed. |
| Budgeted context assembly | Step 15 truncated the constraints and quality collapsed. |
| Checkpointing and resume | A deploy killed a forty-minute run at minute thirty-eight. |
| Error classification | The same failing call ran six times, identical arguments. |
| Token, dollar, and time budgets | One malformed request outspent a normal day of traffic. |
| Trajectory export and replay | A reported failure could not be reproduced. |
| Approval gates on writes | A correct action still needed human authorisation. |
Should you adopt a framework or write your own loop?
Adopt a framework for control flow, and own your context assembly. Control flow is well-understood machinery worth inheriting: state, checkpointing, resumable runs, streaming, human-in-the-loop interrupts. Context assembly is where your domain judgement lives, and a library that concatenates history for you takes the control you need most.
| Option | What you inherit | What you accept | Choose it when |
|---|---|---|---|
| Your own thin loop | Total control of the window, one file to read when it breaks | You write persistence, retries, tracing, streaming, interrupts | Few tools, one or two workloads, strict context control |
| LangGraph | Graph state, checkpointing, resumable and interruptible runs | A framework in every stack trace, plus upgrade churn | Branching flow, long runs, approval steps, many workloads |
| CrewAI | Role, task, and crew scaffolding that runs quickly | Weaker visibility into each agent’s window | Prototypes, and work mirroring a human division of labour |
| LlamaIndex | Ingestion, indexing, and retrieval machinery with agent wrappers | A retrieval-centric structure you build the agent inside | Retrieval is the hard part, not the loop |
| A vendor harness | The fastest path to working, with tracing and evaluation attached | Portability, and the vendor’s opinions about context | Speed beats portability and you already run there |
Debuggability is the tradeoff teams underweight. At three in the morning the question is always the same: what exactly went into that model call. A thin loop answers it by reading one function; a framework answers it only if it exposes the assembled prompt per step. Before adopting anything, break a run on purpose and try to retrieve the exact prompt text of the failed step. If you cannot, you bought speed with blindness.
Lock-in is manageable if you are deliberate about ownership. Your tool contracts, evaluation suite, trajectory schema, and context assembly rules are the durable assets. Keep them as plain functions and plain data, and migration becomes a rewrite of the choreography rather than of the system. The teams that cannot leave put business logic inside framework-specific constructs.
What does the rest of this course cover?
The next nine posts build the harness above, one component group at a time, in dependency order.
LH2 closes the loop with stop conditions and oscillation detection. LH3 and LH4 feed it: context as a budget, then memory that does not go stale. LH5 and LH6 let it act, through typed tool contracts and structured output you can validate. LH7 through LH10 prove and protect it, with trajectory evaluation, a named failure taxonomy, cost and latency control, and defences against prompt injection.
Where this goes wrong
A model upgrade made the agent worse. The symptom is a regression after a change nobody thought was risky. The cause is a prompt tuned around the previous model’s habits, with no trajectory-level evaluation to catch drift. Build the evaluation suite first, then upgrade, then compare runs instead of opinions.
The agent reports success and nothing changed. The symptom is a satisfied summary over an unchanged source system. The cause is verification that trusts a return code or the model’s narration. The fix is a step that queries the system of record and compares observed to intended in code.
Runs do not terminate. The symptom is a step count climbing until an unrelated timeout kills it. The cause is stop conditions written as prompt instructions. Add a counter, a deadline, and a no-progress check inside the loop, each logging a named exit reason.
Nobody can reproduce the failure. The symptom is a reported bug with no path back to it. The cause is logging that keeps the final answer and discards each step’s inputs. Persist the full trajectory, including the assembled prompt, and build the replay path before you need it.
Common questions
Is a harness the same thing as an agent framework?
No. A harness is the set of components surrounding a model call, and a framework is one way to implement some of them. You can build a harness with no framework, and you can adopt a framework and still lack verification, budgets, and trajectory logging, which means you still do not have a production harness.
Do I need a harness if I only make one model call?
No. A single call with no dependent follow-up is a call site, and calling it an agent adds cost without capability. The threshold is the second call whose input depends on the first call’s result, because from there you need state, an error path, and a stop condition.
Which harness component should I build first?
Verification, then budgets and stop conditions, then trajectory logging. Verification turns silent failures into visible ones, budgets turn unbounded cost into a known ceiling, and trajectory logging makes everything after that debuggable.
How much of my token spend is harness overhead?
Enough to measure per step rather than per run. System instructions and tool schemas are usually resent every step, so their cost multiplies by step count, which is why long runs are dominated by content the user never sees.
Can a stronger model let me skip harness work?
It can save you some prompt engineering, and it cannot supply the components. Verification, budgets, persistence, and error recovery are properties of code that runs between model calls, and no improvement inside a single call creates them. As models improve, a good harness gets more valuable, because less capability is lost between reasoning and outcome.
Next steps
Read prompt looping and how to design a loop that closes next, because the loop turns the other eight components into a system, and it is where most teams find out their agent was a chain all along.
For a concrete implementation of every component here, Hermes Agent, a self-hosted agent harness is a working example you can install and read. Earlier field notes on the same territory sit in the Signals archive.
Then run the audit. Sort your last twenty failed runs with the matrix above and fix the harness-bound ones in the order given. We organise harnesses by these component boundaries in production, including the aviation intelligence work behind AeroFarr, precisely so an incident review names a component instead of blaming the model.