Choosing an AI Agent Framework: What Actually Matters
Any article that ranks agent frameworks by name is out of date before it is indexed. Libraries in this space rewrite their core abstractions every couple of releases, and the one that looks best in a benchmark today may have moved on by the time your project ships.
So this guide does something more durable: it lists the eight questions that actually determine whether you will still be glad about your choice in six months, and explains what each answer costs you. Take it to whatever the current shortlist is and you will get a decision you can defend.
The eight questions, in the order they matter#
- Can I read the loop? If you cannot find the file where the model output is turned into a tool call, you cannot debug a bad run.
- What happens on a tool failure — does it surface to me, or is it retried invisibly with a different prompt?
- Is my prompt the framework’s prompt? Hidden system text you did not write will surprise you in an audit.
- Can state be persisted and resumed, or does a crash lose the run?
- How are tools defined, and can I reuse those definitions outside this framework?
- What is the upgrade story — have the core abstractions been renamed in the last two releases?
- Can I swap models without swapping frameworks?
- What does it add to a cold start and to each turn?
Three broad categories, three different bargains#
| Category | You get | You give up | Sensible when |
|---|---|---|---|
| Provider SDK plus your own loop | Total visibility, minimal dependencies | You write retries, state, persistence yourself | One agent, few tools, high debuggability need |
| Orchestration library | Durable state, branching, retries, resumption | Some visibility; upgrade churn | Long-running or multi-step workflows |
| Managed agent platform | Hosting, tracing, evaluation, UI | Portability; per-seat or per-run pricing | Small teams, standard tasks, fast proof |
Write the parts that are yours to keep#
Whatever you choose, four assets should live in your own repository in a form no framework owns: the prompts, the tool definitions and their JSON schemas, the evaluation set, and the trace format. These are the things that took real work to get right. If they are expressed as plain data and thin adapters, changing framework is a day of adapter work. If they are expressed as framework decorators and inherited classes, changing framework is a rewrite — and you will therefore not change, even when you should.
The evaluation nobody does but everybody should#
Before committing, build the same small agent twice: once on your shortlist favourite and once on the provider SDK with a hand-written loop. Give both the same three tools and the same ten test cases. You are not measuring accuracy — both will be similar. You are measuring how long it took, how readable the trace is, and how easy it was to find out why case seven failed. That afternoon has saved every team we know of far more than it cost.
Keep the hand-written version. It becomes your reference implementation when you need to prove whether an oddity comes from your prompt or from the framework.
Signals that you have outgrown your choice#
- You are reading the framework source more often than your own.
- You maintain a patch or a fork to get behaviour you needed.
- Upgrades are deferred because of breaking renames, and you are two majors behind.
- Half your prompt exists to counteract text the framework injects.
- Tracing requires a bespoke exporter because the built-in one hides tool arguments.
Frequently asked questions
Do I need a framework at all for a first agent?
No. A first agent with three tools is a loop, a schema list and a stopping condition. Building it by hand once teaches you what a framework would be doing on your behalf, which makes the later choice much better informed.
Is a managed platform a trap?
Not if you keep your prompts, tool schemas and evaluation set portable. Platforms are genuinely fast to a working result. The risk is not the platform, it is letting your intellectual property exist only as configuration inside it.
How much does framework choice affect accuracy?
Far less than people expect. Accuracy comes from tool design, grounding and evaluation. Frameworks affect development speed, debuggability and operational features — which matter, but not in the way benchmark tables imply.
ai agent frameworkbest agent frameworksagent library comparisonllm orchestrationchoosing agent stack
