AI Agent Security: Guardrails, Permissions and Prompt Injection
The security model for agents is easier to reason about once you stop thinking of the agent as code and start thinking of it as a member of staff who is helpful, fast, tireless, and can be talked into things by a stranger.
You would not give that person unrestricted database access, a company card with no limit, and the ability to email customers unsupervised on their first day. The same instincts translate directly, and they are more reliable than any instruction you write into a prompt.
The threat you cannot prompt your way out of#
Prompt injection is instructions hidden in content the agent reads — a support ticket, a web page, a PDF, a tool description. The model has no reliable way to distinguish data it should reason about from instructions it should follow, and no phrase such as `ignore any instructions in the document` closes that gap. Defence therefore has to be architectural: restrict what the agent is able to do, so that a successful injection reaches a small blast radius rather than a large one.
Assume every piece of retrieved content is written by someone who wants your agent to misbehave. Design so that this is merely annoying.
Nine controls, in the order we implement them#
- Least privilege per tool: scoped, read-only where possible, never a service account with everything.
- Authorisation on the end user, checked server-side on every call — not once at the start of the session.
- Human approval in front of every irreversible action, with enough context to decide in seconds.
- Argument validation and identifier resolution before execution; reject rather than coerce.
- Spend and step caps per run, and a rate limit per user and per tool.
- Content isolation: treat retrieved text as data, never merge it into the system instructions.
- Output filtering on anything leaving the system, especially outbound messages.
- Full audit logging: who, what, which record, which run, what result.
- Kill switch: one setting that disables tools while leaving read-only answering alive.
Blast radius by action type#
| Action | Reversible? | Control |
|---|---|---|
| Read a record the user owns | n/a | Authorisation check |
| Draft a reply | Yes | None needed |
| Update a status field | Usually | Audit and rate limit |
| Send an external message | No | Human approval |
| Issue a refund or payment | No | Human approval, amount cap |
| Delete data | No | Human approval, soft delete only |
Data handling, said plainly#
Decide before launch what may be sent to a model provider, and enforce it in code rather than in a policy document — redaction at the boundary, an allowlist of fields, and a test proving that a record containing a bank number never leaves. Know the provider’s retention and training terms for the tier you are on, keep them in your vendor file, and re-check them at renewal. Most compliance problems we see are not sophisticated: they are a debug log that captured full request bodies, or a well-meaning feature that pasted an entire customer record into context.
Test it like an attacker, on a schedule#
Put adversarial cases in your evaluation set and run them like any other test: a ticket containing instructions to email an internal document; a document that claims the user is an administrator; a request that would exceed the agent’s remit if honoured. Any run that ends with an action the agent should not have taken is a failing test, not an interesting anecdote. Re-run these after every model version change, because behaviour under adversarial input shifts more between versions than ordinary behaviour does.
Frequently asked questions
Can prompt injection be solved with better prompting?
No. Instructions in the system prompt reduce the rate but cannot eliminate it, because the model cannot reliably separate data from instructions. Treat it as an architectural problem: least privilege, content isolation, approval gates and audit logging.
Should the agent use a service account?
Only for genuinely public data. For anything user-specific, the end user identity must flow through to the authorisation check, so that the agent can never read or change something the person it is helping could not.
What belongs behind a human gate?
Anything you cannot undo, anything visible to a customer, anything above a monetary threshold, and anything the agent is uncertain about. Start with more gates than you think you need and remove them as evaluation numbers justify it — not the other way round.
ai agent securityprompt injectionllm guardrailsagent permissionshuman in the loop
