How AI Agents Work: The Loop, Step by Step
Agents look like magic in demos and like plumbing in production. The reason is that the interesting part is not the model output but the loop that consumes it, and the loop is short enough to read in one sitting.
This guide walks a single request through that loop end to end: what the model sees on each turn, what your code does with the result, how a failing tool comes back, and what makes the loop stop. If you can narrate this for your own system, you can debug it. If you cannot, no amount of prompt tuning will make it reliable.
One turn of the loop, in order#
- Assemble context: the goal, the tool definitions, relevant retrieved facts, and a trimmed history of what has already happened.
- Ask the model for the next step. It either answers directly or requests a tool call with arguments.
- Validate the arguments before doing anything — types, ranges, and whether this caller is allowed to touch this record.
- Execute the tool. Catch failures and turn them into short, factual messages rather than stack traces.
- Append the call and its result to the history, then check the stopping conditions.
- Repeat, or return the final answer with whatever the agent actually did.
What the model can and cannot see#
The model has no memory of the previous turn beyond what you put back into the context. That single fact explains most confusing agent behaviour. If the agent forgets a constraint mentioned four steps ago, it is because your history trimming dropped it. If it retries the same failing call three times, it is because the failure message did not say why it failed in words the model could act on. Context assembly is not preamble to the interesting work; it is the interesting work.
Write your tool errors as instructions, not as diagnostics. Not `HTTP 404` but `No customer with that ID. Ask the user to confirm the order number.`
Planning: explicit or emergent#
There are two respectable ways to get a plan. Emergent planning lets the model choose one step at a time with no plan document — simple, resilient, and prone to wandering on long tasks. Explicit planning asks for a numbered plan up front, then executes it step by step, re-planning only when a step fails. Explicit planning is easier to audit and much easier to show a user, at the cost of being brittle when reality diverges from step three. For tasks under about five steps, emergent is usually enough; beyond that, an explicit plan pays for itself in traceability.
Stopping: the part demos never show#
| Condition | Typical setting | What happens when it trips |
|---|---|---|
| Step cap | 8–15 tool calls | Return partial work with an explanation |
| Spend cap | A fixed cost per run | Stop and log for review |
| Wall clock | 30–120 seconds for interactive use | Hand back with what is known so far |
| Repeat detection | Same call and arguments twice | Force a different branch or stop |
| Human gate | Any irreversible action | Pause and request approval |
Reading a trace when something goes wrong#
A trace is the ordered record of every context, decision, call and result in one run. It is the only debugging tool that matters, and the first thing to build. When an agent misbehaves, the question is never why the model is bad; it is which turn first went wrong and what the model could see at that moment. Nine times out of ten the answer is boring: a tool returned an empty list and said nothing about it, a stale fact stayed in the context, or a permission error was phrased as a generic failure and the model treated it as retryable.
Frequently asked questions
How many steps should an agent take before stopping?
For interactive tasks, a cap of eight to twelve tool calls covers almost everything legitimate; a run that needs more is usually stuck. Batch tasks can go higher, but pair a higher cap with a spend cap so a loop cannot be expensive as well as long.
Should the agent plan first or decide step by step?
Short tasks do fine deciding one step at a time. Once a task reliably takes more than five steps, an explicit plan makes the run auditable and lets you show progress to a user — re-plan on failure rather than following a stale plan off a cliff.
Why does my agent repeat the same failing call?
Almost always because the failure message contains no actionable information. Return short, plain-language errors that state what was wrong and what a sensible next step would be, and add repeat detection so an identical call with identical arguments cannot happen twice in one run.
how ai agents workagent loopreason act observetool calling loopagent architecture
