AI Agent Architecture: The Patterns That Hold Up in Production
Agent architecture discussions usually start at the wrong end, with a diagram of boxes named after concepts. The useful version starts with the failure you are trying to prevent, because every pattern below exists to stop one specific bad afternoon.
These six are the ones we keep reaching for. They compose: a production agent is typically a bounded loop with a tool gateway, two memory tiers and a human gate, and a critic pass only where the cost of a wrong answer justified another model call.
1. The bounded loop#
The base case, and the one you should default to. A single loop over a small tool set with explicit stopping conditions: a step cap, a spend cap, repeat detection and a wall-clock limit. Its virtue is a linear trace an engineer can read from top to bottom. Every other pattern here is an addition to this, not a replacement for it.
If you cannot draw your agent as a loop with a list of exits, you do not yet have an architecture; you have a prompt with ambitions.
2. Planner–executor with re-planning#
For tasks that reliably take more than about five steps, ask for an explicit numbered plan first, execute the steps, and re-plan when a step fails rather than continuing down a stale plan. The benefit is not accuracy — it is that a human can see what the agent intends before it acts, which makes both approval and debugging tractable. The cost is one extra model call, plus the discipline of treating the plan as revisable rather than sacred.
3. The critic pass#
A second model call reviews the draft answer or the proposed action against the goal and the retrieved evidence, and can send it back once. This catches a meaningful share of confident-but-unsupported output. Use it where a wrong result is expensive and one extra call is not: outbound customer messages, financial summaries, code changes. Do not use it everywhere — it doubles cost and latency, and on easy tasks it mostly agrees with itself.
| Pattern | Extra model calls | Buys you | Skip when |
|---|---|---|---|
| Bounded loop | 0 | Traceability, cost control | Never — this is the base |
| Planner–executor | 1–2 | Visible intent, auditability | Tasks under five steps |
| Critic pass | 1 per checked output | Fewer unsupported claims | Cheap, reversible outputs |
| Tool gateway | 0 | Permissions, audit, rate limits | Prototypes only |
| Memory tiers | 0–1 | Relevance at long context | Short single-turn runs |
| Human gate | 0 | Irreversible actions stay safe | Nothing irreversible exists |
4. The tool gateway#
Do not let the agent call your systems directly. Put one layer in front of every tool that does four things: validates arguments against a schema, checks that this end user may touch this record, applies a rate limit, and writes an audit line with the run identifier. This is the single highest-value piece of infrastructure in an agent system, and it is ordinary code that takes days rather than weeks. It also means switching agent frameworks never touches your security posture.
5. Memory in tiers#
One undifferentiated conversation history is the most common cause of an agent that gets worse as a run continues. Separate what you carry: the goal and constraints, which never get trimmed; the recent turns, kept verbatim; the older turns, compressed into a short factual summary; and retrieved knowledge, fetched fresh per step and never accumulated. Long contexts are expensive and attention is finite — carrying everything is not thoroughness, it is dilution.
6. The human gate#
Every irreversible action sits behind an explicit approval step with enough context for a person to decide in seconds: what will happen, to which record, why the agent believes it should, and what it will do if declined. The gate is a product feature, not a limitation — it is what lets you ship an agent into a system where mistakes are expensive, and it is what you remove selectively once your evaluation numbers earn it.
Frequently asked questions
Do I need all six patterns?
No. Start with the bounded loop and the tool gateway; those two are close to mandatory for anything touching real systems. Add the human gate the moment an irreversible action appears. The rest are earned by specific failures you can point to in a trace.
Does a critic pass really improve accuracy?
On tasks where the model can produce a plausible but unsupported answer, meaningfully yes — particularly when the critic is given the retrieved evidence and asked to check claims against it. On simple lookups it mostly agrees with the first answer and doubles your cost.
Where should the tool gateway live?
In your own service, between the agent and your systems, with the end user identity flowing through it. If it lives inside the agent framework, you will re-implement it the next time you change frameworks, and your security review will start again from zero.
ai agent architectureagent design patternsplanner executortool gatewayagent memory design
