Monitoring AI Agents in Production: What to Log and What to Alert On
A conventional service is healthy when it responds quickly and does not throw. An agent can do both while being completely wrong — every request answered in two seconds, every one of them citing a policy that was withdrawn in March.
So agent monitoring is a different discipline. It rests on two things: a trace of every run detailed enough to reconstruct what happened, and a small set of behavioural metrics whose movement means something. Everything else is decoration.
What a useful trace contains#
- A run identifier attached to every log line, model call and outbound request.
- The exact context sent to the model at each step, or a hash plus the assembled parts.
- Every tool call with arguments, result, duration and outcome.
- Model name and version per call, plus token counts.
- The stopping reason: completed, step cap, spend cap, human gate, error.
- The final output, and whether a human later corrected or reversed it.
Six metrics that actually move#
| Metric | Watch for | Usually means |
|---|---|---|
| Task success rate | Sustained drop | Model change, data drift, upstream API change |
| Steps per run | Creeping up | Tool errors being retried; degraded retrieval |
| Tool error rate by tool | Spike on one tool | Upstream break — not an agent problem |
| Human intervention rate | Rising | Trust falling, or a new request category |
| Unsupported claim rate | Any rise | Retrieval failing quietly |
| Cost per completed task | Rising with flat volume | Context bloat or extra retries |
Alert on behaviour, not only on errors#
An agent rarely fails loudly. It degrades: slightly more steps, slightly more retries, slightly more escalations, and one morning it is answering from stale documents. Alert on rate of change over a rolling window rather than absolute thresholds — steps per run up 30% week on week is a real signal, while an individual eight-step run is not. Pair every alert with the trace of a representative run, because an alert nobody can investigate quickly becomes an alert everybody mutes.
Sample and read real runs, every day#
No dashboard replaces reading. Pick a handful of runs daily — a couple of successes, every escalation, every capped run — and read them end to end. Every serious problem we have found in a production agent was visible in a trace before it was visible in a metric. Rotate who does the reading; the person who wrote the prompt is the least likely to notice what it is doing wrong.
Add anything surprising you find to the evaluation set the same day. That habit is what turns monitoring into improvement rather than observation.
Logging without collecting things you should not#
Traces contain customer data by construction. Redact identifiers at the point of logging rather than in a later job, set a shorter retention on full contexts than on metrics, and keep tool arguments — which is where personal data concentrates — under the same access controls as the underlying system. Storing everything forever because it might be useful is how a monitoring project becomes a compliance finding.
Frequently asked questions
How long should I keep full traces?
Long enough to debug and to satisfy audit needs — commonly 30 to 90 days for full contexts, with metrics and outcome summaries kept much longer. Set the retention deliberately, because these records contain whatever your users typed.
What is the single most valuable alert?
A rise in escalations or human corrections. It is the earliest honest signal that behaviour has drifted, and unlike accuracy it needs no labelling — your users and operators are grading the agent for free.
Do I need a specialised observability tool?
Not to start. A structured trace table you can query covers most needs. Specialised tools help when you want side-by-side run comparison, evaluation integration and prompt versioning in one place — adopt one when reading traces has become a daily job for several people.
ai agent monitoringllm observabilityagent tracingproduction ai metricsagent drift
