Lesson 1.2: The agent loop: observe, decide, act
Follow an agent through its loop, and learn why small per-step errors compound, why context grows with every turn, and why every loop needs a limit.
By the end of this lesson you will be able to describe the loop that every agent runs, trace a real task through it one step at a time, explain why an agent that is right 95 times in 100 at each step finishes a ten-step job correctly only about 60 times in 100, and estimate how the amount of text the model reads grows as a run gets longer. You will work out that a ten-step run which adds 500 tokens per step reads 42,500 tokens in total against 20,000 if nothing accumulated, and decide where a loop needs stop rules. You should have read Lesson 1.1, which separates chatbots, workflows and agents. The mathematics is repeated multiplication and a running sum.
Every agent, however sophisticated, is a loop. Frameworks, diagrams and product names differ, but underneath there is a cycle that repeats until a stop condition is met. If you understand this cycle well, you can read the design of any agent system, predict where it will fail and make sensible choices about its limits. This lesson takes the loop apart.
The loop in one sentence
An agent looks at the situation, decides what to do, does it, and looks at the result, then goes round again. The usual names for the three parts are:
1. Observe: gather what is known. This is the goal, the instructions, everything that has happened so far, and the result of the last action.
2. Decide: the model reads all that and chooses the next move. The move is either a request to use a tool or a decision that the work is finished.
3. Act: the surrounding program carries out the move, for instance by running a search, and records the result.
Then the result becomes part of what is observed, and the cycle repeats. Some writers draw it as "think, act, observe". The names vary and the structure does not.
It is worth being exact about who does what, because it prevents a common confusion. The model never runs anything. It reads text and produces text. When it "uses a tool", it writes a structured request, for example "call search with the query late fee reversal policy". The program around the model, usually called the harness or the runtime, reads that request, runs the real function, and adds the result to the text the model sees next. So the loop is a partnership: the model supplies judgement, and the harness supplies action, limits and memory.
One task, step by step
Here is a complete trace for a task given to an agent that supports an accounts team: "Find out why invoice 8841 was rejected and tell me what to fix." The agent has three tools: getinvoice, getrules and finish.
Step · What the model sees · What the model decides · What the harness does · Result1 · The goal and instructions · Call getinvoice for 8841 · Fetches the record · Status rejected, code E12, amount 48,000
2 · Goal, plus step 1 · Call getrules for code E12 · Looks up the rule · "E12: purchase order number missing or invalid"
3 · Goal, plus steps 1 and 2 · Compare the invoice with the rule · Nothing, this was reasoning with no tool · The invoice has no purchase order field
4 · Goal, plus steps 1 to 3 · Call finish with a summary · Returns the answer · "Rejected because the purchase order number is missing. Add it and resubmit."
Look at what the model sees at each step. At step 1 it sees little. At step 4 it sees everything that happened before. This is how an agent has memory within a run: nothing is stored inside the model between calls, so the harness re-sends the whole history every time. That fact has large consequences, and the second half of this lesson explains two of them. First, though, notice the three kinds of exit from the loop.
How a loop ends
A loop needs to end, and it can end in several ways. A well-designed agent has all of them.
Success: the model decides the goal is met and calls finish. This is the normal exit.
Failure by choice: the model decides it cannot complete the task, for example because a record does not exist, and says so, ideally handing the case to a person.
A step limit: the harness stops the loop after a maximum number of steps, such as 15, whatever the model wants. This protects against endless wandering.
A budget limit: the harness stops when the run has used too much time, text or money.
A safety stop: the harness stops when the model asks for something that is not allowed, or when a person presses a stop button.
Without the last three, a confused agent can circle for as long as you let it. A model that repeatedly makes the same failing call, hoping for a different result, is a familiar failure, and only the harness can end it reliably, because the model is the very part that is confused.
Reading a trace to find the wrong turn
When an agent fails, the first tool is its trace: the record of every step. Reading a trace is a skill, and it is mostly looking for the first step that went wrong, since later steps usually inherit the damage. Here is a failed run of the invoice task.
Step · Decision · Result · Verdict1 · Call getinvoice for 8814 · Invoice found, status paid · Wrong number: the request said 8841
2 · Call getrules for code none · Error: no rejection code · Harmless, but caused by step 1
3 · Call getinvoice for 8814 again · Same record · Wasted step, a sign of confusion
4 · Finish: "Invoice is paid, nothing to fix" · Answer returned · Confident and wrong
The root cause is step 1, a swapped pair of digits in the tool argument. Steps 2 to 4 are consequences. Three fixes would each have prevented it: a tool that echoes back the customer name and amount so the model can notice a mismatch, a check that the invoice number in the answer matches the one in the request, and a rule that a paid invoice cannot be the subject of a rejection question. Notice that none of the fixes is "tell the model to be more careful". They change the loop around the model, which is where most reliability comes from.
Why errors compound
Now to the central fact about loops. A single model call is imperfect. Suppose that at each step the agent makes a sound decision with probability 0.95, meaning that 5 steps in 100 involve a bad choice such as the wrong tool, the wrong argument or a misread result. That sounds excellent. Now consider a task that needs ten steps, each of which must be right for the whole to be right.
The chance that all ten are right, if the steps are independent, is 0.95 multiplied by itself ten times: 0.95^10 = 0.599. A system that is right 95 per cent of the time at each step completes a ten-step task correctly only about 60 per cent of the time. At five steps it is 77 per cent, at fifteen 46 per cent, and at twenty 36 per cent.
Predict first: a step is right 99 times in 100, much better than 95. Roughly how often will a twenty-step task be completed correctly: about 99 per cent, about 90 per cent or about 80 per cent? Decide before reading on.
About 80 per cent. 0.99^20 = 0.818. Even a step that is right 99 times in 100 loses almost a fifth of twenty-step runs. And a step that is right 90 times in 100 completes a ten-step task only 35 times in 100, and a twenty-step task 12 times in 100. This is why long tasks are the hardest for agents, and why a lot of agent engineering is about keeping runs short, making each step more reliable and catching errors before they spread.
The remedies follow from the arithmetic.
Fewer steps. Combine steps, or do some of them in code. A task that can be done in five steps instead of ten raises the 95 per cent case from 60 to 77 per cent.
More reliable steps. Clear instructions, well-designed tools and simpler choices raise the per-step accuracy, a theme of Module 2.
Checks that catch errors. Suppose a check catches half of the wrong steps. A step that was 95 per cent right now ends up 97.5 per cent right, and the ten-step success rate rises from 60 to 78 per cent. Checks can be code, such as validating that a date is a real date, or another model call that reviews the work.
Recovery. A good agent notices a failed tool call and tries again or tries another route, rather than carrying on with a wrong assumption.
Splitting work. Handing part of the job to a sub-agent with a smaller task, covered in Module 5, keeps each run short.
The independence assumption is a simplification. Real errors cluster, since one misunderstanding early on can make every later step wrong, and an agent can also recover from some errors. But the direction is right, and the numbers are a useful warning: per-step accuracy that looks high is not enough for long tasks.
Why the loop gets heavier with every turn
The second consequence of re-sending the history is cost. At every step the model reads the instructions, the goal, and everything that has happened so far. The text it reads grows with each turn.
Suppose the instructions and goal take 2,000 tokens, and each step adds about 500 tokens of tool results and model output. (A token is the unit in which models measure text, roughly three quarters of an English word.) At step 1 the model reads 2,000 tokens. At step 2 it reads 2,500. At step 10 it reads 6,500. The total read across a ten-step run is 2,000 + 2,500 + 3,000 and so on up to 6,500, which is 42,500 tokens. If nothing accumulated and each step read only 2,000, the total would be 20,000. The history more than doubles the reading.
The growth is not linear in the total. For a twenty-step run the total is 135,000 tokens, against 40,000 without accumulation, which is 3.2 times the ten-step total of 42,500 although the run is only twice as long. Doubling the steps more than triples the reading, because each new step carries all the old steps with it.
Two practical conclusions follow. First, cost and delay rise faster than the number of steps, so long runs are disproportionately expensive. Second, there is a hard ceiling. A model can only read a limited amount of text at once, called its context window, and a run that keeps adding to the history will eventually hit it. Even before it does, models tend to use very long contexts less well, missing details buried in the middle. Good agents manage this: they trim tool results to what matters, summarise older steps, and move finished sub-tasks into separate runs. Module 4 is devoted to these techniques.
What happens at each step in detail
It helps to look inside one turn of the loop, because several design choices live there.
1. The harness builds the prompt. It assembles the instructions, the tool descriptions, the goal and the history into one block of text.
2. The model produces a response. The response is either plain text, a tool request or both. Models can be asked to give a short note of their reasoning before the request, which often helps them choose well.
3. The harness parses the response. If it is a tool request, the harness checks that the tool exists and that the arguments are valid. A malformed request is not run. Instead, an error message goes back to the model so that it can correct itself.
4. The harness may check permissions. Some actions, such as paying money or deleting records, can require a person's approval before they run.
5. The harness runs the tool, with a time limit in case it hangs.
6. The result is added to the history, often shortened if it is long.
7. The harness checks the stop rules and either ends the loop or goes round again.
Every one of these steps can be improved by engineering, and every one is a place where things go wrong. A tool that returns a 50,000-word document floods the context. A request that names a nonexistent tool wastes a step. A missing time limit lets one slow call stall the whole run.
The loop in code
Here is the loop again, this time recording the quantities from this lesson: the number of steps, the text size at each step, and which stop rule ended the run. The model is a stand-in so the code runs anywhere.
python
def standinmodel(history, goal):
"""Pretend model: asks for two lookups, then finishes. Replace with a real model call later."""
if len(history) < 2:
return {"tool": "lookup", "args": {"n": len(history) + 1}}
return {"tool": "finish", "args": {"answer": "done"}}
def lookup(n):
return "result " + "x" 400 n # a result that is a bit longer each time
TOOLS = {"lookup": lookup}
MAXSTEPS, MAXTOKENS = 15, 12000 # the harness enforces both limits
def tokens(text):
return len(text) // 4 # a rough rule: about four characters per token
def run(goal, instructions="You are a careful assistant. " 40):
history, readtotal = [], 0
for step in range(1, MAXSTEPS + 1):
prompt = instructions + goal + "".join(str(h) for h in history)
readtotal += tokens(prompt) # the whole history is read again at every step
if readtotal > MAXTOKENS:
return "stopped: token budget", step, readtotal
action = standinmodel(history, goal)
if action["tool"] == "finish":
return "finished: " + action["args"]["answer"], step, readtotal
if action["tool"] not in TOOLS: # a request for a tool that does not exist
history.append({"error": "unknown tool " + action["tool"]})
continue
runtool = TOOLS[action["tool"]]
history.append({"tool": action["tool"], "result": runtool(action["args"])})
return "stopped: step limit", MAXSTEPS, readtotal
print(run("Find the answer"))
Even in this toy, the structure shows: the prompt is rebuilt from the full history each time, the total read grows faster than the step count, and two limits protect the run.
Common mistakes
Believing the model remembers between steps. It does not. The harness re-sends the history, and anything left out is gone.
Trusting a high per-step accuracy. Multiply it across the steps. Ten steps at 95 per cent is only about 60 per cent overall.
No step or budget limit. A confused agent will loop. Limits belong in the harness, not in the instructions.
Letting tool results grow without bound. A huge result floods the context and slows or confuses later steps.
Treating the model's description of an action as the action. Only the harness running a tool changes anything.
No error path. If a tool fails or a request is malformed, the model must be told clearly so it can recover.
Measuring only the average run. The long and failing runs drive cost and complaints.
Making the task longer than it needs to be. Merge steps and use code for the fixed parts.
Key takeaways
Every agent is a loop: observe, decide, act, repeat until a stop condition. The model chooses, and the harness runs tools, applies limits and keeps the history.
The model has no memory between calls. The harness re-sends everything each time.
A loop needs exits: success, a chosen failure, a step limit, a budget limit and a safety stop.
Errors compound. With 95 per cent per-step accuracy, a ten-step task is fully right about 60 per cent of the time, and a twenty-step task 36 per cent.
The history makes long runs heavy. With 2,000 starting tokens and 500 added per step, ten steps read 42,500 tokens against 20,000 without accumulation, and twenty steps read 135,000.
Remedies: fewer steps, more reliable steps, checks that catch errors, recovery paths, trimmed results and splitting work.
Check yourself
1. In the loop, who runs a tool: the model or the harness?
The harness. The model only writes a request. The harness runs the real function and adds the result to the history.
2. A step is right 98 times in 100. What is the chance a ten-step task is fully right?
0.98^10 = 0.817, about 82 per cent.
3. An agent starts with 1,000 tokens of text and adds 400 per step. How many tokens does it read at step 6, and in total over 6 steps?
At step 6: 1,000 + 400 × 5 = 3,000. Total: 6 × 1,000 + 400 × (6 × 5 ÷ 2) = 6,000 + 6,000 = 12,000.
4. Why is a step limit set in the harness and not in the instructions to the model?
A confused model may ignore or forget an instruction. The harness runs outside the model and can always stop the loop.
5. A check catches 60 per cent of wrong steps. A step was 90 per cent right. What is its effective accuracy?
The error rate falls from 10 per cent to 4 per cent, so the step is 96 per cent right.
Try this
Set aside forty-five minutes.
1. Take a task you might give an agent, such as finding out why a customer's order was delayed. Write the likely steps, one line each, and the tool each would use.
2. Trace the loop for the task in a table with the columns used in this lesson: what the model sees, what it decides, what the harness does and the result.
3. Count the steps. Using the reliability simulator, find the chance of a fully correct run if each step is 95 per cent right, and again with a check that catches half of the errors.
4. Estimate the text added per step. Use the context simulator to find the total reading for your run and the multiple compared with no accumulation.
5. Write the stop rules for your agent: a step limit, a budget limit, and what should happen on each exit.
6. Name the step you consider most likely to go wrong, and one change that would make it more reliable.
7. Run the code from this lesson. Change the stand-in model so that it never finishes, and watch which limit stops it.
A good result is a traced loop, a reliability estimate with and without a check, a rough token total, and a written list of stop rules. The next lesson asks the question that comes before any build: how much freedom should an agent have, and when should you not use one at all?
Tags: Agentic AI Agent loop Reliability Context window AI agents