Guarding the Way: How to Actually Evaluate an AI Agent Runtime
Every AI agent runtime faces the same core problem. The model proposes actions, and something has to decide which ones execute. Every runtime claims to have a layer for this. The question that actually matters is whether that layer is the only path from model output to tool execution, and who enforces the word only. We build and study these systems, and the short version of what we have learned is this: you cannot evaluate an agent runtime by reading its architecture documents. In a fast-moving codebase, the docs describe intent. The gap between intent and the default execution path is where the next generation of agent security incidents will live.
The architecture in the docs is not always the one that runs
We recently spent an afternoon tracing one major open-source agent runtime, a serious project with dozens of crates and releases every couple of weeks. Its architecture documents describe capability hosts, time-scoped approval leases, and a policy resolver where constraints can only reduce authority. On paper it is a strong design. In practice, the new capability-gated engine that implements all of that ships disabled by default, behind an environment flag. The code is real, tested, and shipped in the binary. It just is not the path your message takes when you install the product today. The default path is still the older loop the new design was meant to replace.
None of this is hidden. The code comments say parallel deployment right at the top, and this is responsible engineering. Migrating a large codebase toward compiler-checkable boundaries takes quarters, and flags default to off during migrations. The lesson is not that the project is insecure. It is that every README and architecture deck describes the destination, while the binary ships the starting point. If you are betting your business on a runtime, you have to know which one you are actually running.
When authority is just data, it can be forged
In that same runtime, when an action is approved, the approval becomes a lease, a token that represents authority to execute. The lease is defined as an ordinary data structure with public fields, a public constructor, and the ability to be cloned and parsed from JSON. That means any code in any of the dozens of crates can mint a lease, copy one, or deserialize one. Nothing distinguishes a lease issued by the approval resolver from one built by hand three modules away. The authority is plain old data.
Nobody decided to make authority forgeable. Someone needed the lease inside a structure that auto-derived serialization, the derive macro was the path of least resistance, and the compiler had no opinion. When enforcement is convention, every contributor is one convenience away from breaking it, and the build stays green. The better pattern is to make the authority token unforgeable at the type level: a constructor visible only inside the enforcement module, no clone, no copy, no default conversion, with a test that is supposed to fail to compile. If anyone ever refactors the token so that outside code can construct one, the build goes red. Not because a reviewer caught it, but because the compiler did.
MUST is not a type
A third pattern shows up everywhere. A doc comment on a tool dispatcher describes, carefully and correctly, a real security invariant: that certain output must be sanitized at the channel edge before it reaches a user, and that approval checks are skipped because the operations are user-confirmed by definition. Read it as a security engineer and you see thoughtful work that is enforced by nothing. Every future author has to find the comment, understand it, and remember it. The word must is doing the work a type should do. And notice the tell: there are at least two parallel paths that execute tools, and one is said to mirror the other. Mirrors drift. Definitions do not compile.
Five questions to ask any agent runtime
These five questions take an afternoon to answer for a codebase you have never read, and they work on any runtime, including one you are buying. First, can the capability token be constructed outside the enforcement layer? Look for public fields, a public constructor, or auto-derived cloning and deserialization. Each is a forging path. Second, is the policy engine the default path or a flag? Find the environment variable that turns it on. If security is opt-in, the default is the product. Third, how many routes lead from model output to tool execution? One chokepoint, or several paths that mirror each other? Each extra route is a place the rules get re-implemented, or do not.
Fourth, what does the model see when it is denied? If rejections include rule names or pattern text, the policy leaks to the very thing being policed, and the model can probe its way around it. A good runtime returns action not permitted, and nothing else. Fifth, what happens to an unknown tool or an unmatched action? Deny, allow, or ask? Ask-by-default with every tool visible is allow-by-default wearing a seatbelt.
Count the facts, not the promises
Some invariants are too important to be requests. A security invariant in a comment is a request. A security invariant in the type system is a fact. When you pick an agent runtime, or build one, count the facts, not the promises in the documentation. Whatever runtime you are betting on, spend an afternoon tracing one tool call from model output to execution. You will learn more than any README will tell you about what guards the way between a model's suggestion and your production systems.