TL;DR - AI agents in production do not fail loudly. They fail silently, and they tell you the work is done while they are still half-finished. The security question is not whether the model is smart enough. It is whether the run was actually verified. Over the last month, I shipped five deterministic primitives into a personal agent factory (the pifactory) to answer that question without piling on more LLM-as-judge. None of this is exciting. But it works. What you need to do:
I built my first serious AI agent in 2023. It ran in a loop, called tools, and returned an answer. The answer looked right. The model said it was confident. I shipped it. Then I spent two days wondering why the file it claimed to write was empty.
That is the whole post in one paragraph. An AI agent that says it is done is not the same thing as an AI agent that is done. The gap between those two states is where security incidents hide. It is also where audit trails go missing, where compliance evidence disappears, and where a "harmless" misfire quietly becomes a data exposure.
This is a security-first review of the pifactory Tier 1 to Tier 4 implementation I just shipped. The audience I am writing for is you, the IT lead at an SMB or ANZ enterprise who is being asked by your CIO to "evaluate AI agents for the team" and would rather not be the next Frontier Airlines data breach story. I am not pitching a vendor. I am showing my own plumbing.
By The Numbers
| Metric | Number |
|---|---|
| Deterministic primitives shipped | 5 (Tilldone / Team / Damage Control / Purpose Gate / Meta Pi) |
| Read-only expert agents | 8 |
| Writer/orchestrator seats | 1 |
| Tool-call events captured per run | Every state-changing call (write / edit / patch / bash) |
| Test suite | 200 / 200 passing |
| Verifier checks | 17 / 17 passing |
| Worst-case tool-call halt cycle | 1 hour |
| External trust placed in the LLM | Lower than a single shell script |
The Setup
I run a personal software factory called pifactory. It is a chain of AI agents that read a spec, plan the work, build the code, review it, verify it, and ship it. The factory itself is not the interesting part. The interesting part is the surface area it puts on the network, the data it touches, and the trust a security lead has to place in it before turning it on.
Most security write-ups about AI agents cover one of two extremes. Either the agent is treated as a magic black box that needs no review ("it is just an API call"), or it is treated as an active threat that needs to be locked down like a hostile endpoint. Both readings miss the actual risk, which is quieter and more interesting.
The actual risk is this: an AI agent that has been told it is finished its work is, from the perspective of the calling system, finished. The conversation turns. The user moves on. The downstream automation runs. The model has every incentive to declare success, because success is the default state of "I have nothing more to say." When the agent has, in fact, silently dropped a step, or written to the wrong path, or returned success before the verifier ran, there is no alarm. The system believes the agent because the system has no other evidence.
If you have ever shipped a Copilot rollout or sat through a Microsoft MDASH report, you already know this is not hypothetical. The find-to-fix gap in Project Glasswing is the same shape of problem at a different scale. Discovery moves at AI speed. Verification does not. The two speeds compound into a backlog that nobody owns.
This post is about five primitives I shipped into pifactory to bring verification speed closer to discovery speed, without paying for another LLM-as-judge every tool call. The headline lesson: when you cannot trust the model's word that a task is done, give every state-changing tool call a deterministic witness.
What Actually Happened
In July 2026 I shipped five interlocking primitives across what I am calling the Tier 1 to Tier 4 implementation. They are not a wholesale rewrite. They are additive control points around the existing Scout, Plan, Build, Review and Verify chain. Each one addresses a different way an AI agent can silently break a security lead's assumption about what is happening.
I want to walk through each one honestly, including the parts I do not love, because a security lead reading this should be able to lift the pattern into their own agent substrate, whether that is LangChain, AutoGen, a custom harness, or the open-weights local loop you run on your own hardware.
Tier 1: Tilldone makes unfinished work loud
The first primitive is the one I am proudest of, because it is the cheapest and the most useful. I call it Tilldone. It is a completion liveness gate.
The pattern: every state-changing tool call (write, edit, patch, bash with side effects) has to be wrapped in an active task. A task is begun explicitly. A task is closed explicitly. The wrapper checks for an active task before allowing the call. If there is no active task, the call is rejected with a bounded continuation signal - the agent gets a few retries to start a task - and then a hard halt.
Why this matters for security: a worker cannot quietly keep changing state after it has lost track of what it is doing. In my 2023 incident, the agent declared success and then kept writing to disk. Tilldone would have caught that on the second write, because the first write had closed the task. The agent would have had to start a new task, and starting a new task forces the agent to declare what it is doing now. That is not a prompt-level guard. That is a shell-level guard. It cannot be bypassed by clever wording.
The cost is small. A task-state.json file and three wrapper scripts. The benefit is that "done" is a thing the system can check, not a thing the model claims.
Tier 1 cont.: named teams, seat discipline intact
The second primitive makes teams addressable routing frames rather than loose piles of prompts. The shape is scripts/team-dispatch.py, three concrete team definitions (plan-build-review-team, cert-intel-team, content-gen-team), and a team_compatibility field on every agent file.
The security read: a team can coordinate more work, but it does not get a wider tool surface for free. The seat contract underneath remains the boundary. Composition cannot silently bypass it. If you have ever worried about shadow IT forming inside your AI tooling, this is the control that addresses it. The team gets a name and a definition. Every seat in the team declares what it is compatible with. A seat that is not on the list cannot be quietly added by a clever prompt.
This matters more than it sounds. The most common shadow-AI incident I see in 2026 is a model calling a tool that nobody approved. Not a malicious tool. Just an adjacent one the agent thought was helpful. The seat contract makes that visible.
Tier 1 cont.: Damage Control routes failures before retrying
The third primitive sits between Verify and Decide in the chain. The pattern: when a verifier yield fails, a deterministic classifier inspects the failure mode. There are exactly three buckets. Transient (network blip, race condition, retry will probably work). Correctness (the answer is wrong, retry will not help, escalate to a different verifier or route). Unrecoverable (the chain cannot fix this, halt and surface to the human).
The classifier is not an LLM. It is a shell script (failure-control.sh) that reads structured verifier output and applies explicit rules. The script also checks whether a verifier yield satisfies its atomic-claim contract before escalation is allowed. Three successive failures on the same stage halt rather than consume another round. No LLM decides the classification. The dispatch receipt preserves the reason the route was chosen.
The security read: "retry" is not a neutral action. A transient failure deserves a clean-context retry. A correctness failure needs evidence and possibly a second opinion. Burning tokens on a third try when the first two failed for the same reason is not diligence, it is waste. Worse, it is the kind of waste that hides a real correctness failure behind a quiet retry storm. The classifier turns that into a receipt a security lead can audit.
Tier 3: Purpose Gate turns intent into a cost boundary
The fourth primitive exists as a Hermes skill at ~/.hermes/home/skills/tilldone-purpose-gate/. Its job is to take a declared purpose for a dispatch and check that the route and the spend stay aligned with that purpose. If a task is well-scoped to a single expert, the gate refuses to spin up a meta-agent. If a task is genuinely ambiguous, the gate approves the broader route.
This is where the token-responsibly doctrine becomes operational. "Token-responsibly" is my phrase for not paying for the smartest model just because it is available. It is the opposite of TokenMaxxing. If the task is well-scoped, you do not need a meta-agent. If the task is ambiguous, you do. The gate is the operational form of that judgement, not the human's. It is a script that reads the task, reads the route, and decides.
The security read: a Purpose Gate gives you an audit trail for every dispatch that explains why a particular route was taken. That trail is gold for a security review. When the next Claude Mythos or Copilot rollout lands in your environment and someone asks "why did the agent do that," you can answer with a receipt, not a shrug.
Tier 4: Meta Pi adds breadth, with a hard read-only boundary
The fifth primitive is the most architecturally interesting one. Meta Pi is a meta-agent: eight expert agents, each owning one concern (repo scanning, spec interpretation, risk, dependencies, testing, style, precedent, and one more I will leave for the next post), plus one writer/orchestrator that composes their yields.
The load-bearing choice is not the number eight. It is the split. The eight experts are read-only. They cannot write to anything outside their own scratch space. They cannot see one another's mutable state. The writer is the only merge boundary.
The dispatcher (meta-dispatch.py) rejects forbidden tools at the gate. The experts get to inspect, classify, and recommend. They do not get to act. The writer receives their deterministic yields and composes the final output. Readers cannot clobber each other. The writer cannot accidentally read what the readers could not write.
The security read: this is the pattern you want if you are going to let an AI agent touch anything that matters. Multiple read-only views, one write surface, every read receipted. It is the closest I have got to the principle of least privilege applied to an LLM. The model wants to do everything. The harness makes it prove what each piece did, and who authorised it.
The Numbers
The numbers tell a cleaner story than the prose. Here is what shipped, in the order an auditor would ask about it.
| Layer | Question the gate answers | Mechanism | Default action on miss |
|---|---|---|---|
| Tilldone | Is this state-changing call inside an active, declared task? | task-state.json + wrapper script | Reject with bounded continuation, then halt |
| Team-shape | Is this seat compatible with the team that wants to instantiate it? | team_compatibility field on every agent file | Reject seat from team instantiation |
| Damage Control | Is this verifier failure transient, correctness, or unrecoverable? | Deterministic classifier on structured output | Escalate or halt, never silent retry |
| Purpose Gate | Is the route and spend aligned with the declared purpose? | Skill script that reads task + route | Refuse over-scoped dispatch |
| Meta Pi | Is this expert trying to write outside its own scratch space? | Dispatcher rejects forbidden tools | Hard refusal, logged in run receipt |
The verification harness has 200 of 200 tests passing. The Purpose Gate skill has 17 of 17 self-checks passing. Every state-changing tool call in a Tier 1 to Tier 4 run is receipted in a per-tool-event stream that records schema version, timestamp, run ID, stage, tool, declared purpose, purpose alignment, token count, duration, and cost.
That last column matters. Per-tool cost. Without it, you cannot audit token-responsibly. With it, you can show a finance team and a security team exactly where the spend went and whether it bought anything.
Examples From Real Runs
Three short examples, each one a real failure mode the primitive in question was designed to catch.
Example 1: the agent that finished twice. A build agent ran a verification step, declared success, then - because the chain had not yet closed - wrote a configuration file to a path it had already written to. The first write had succeeded. The second write created a duplicate config. Without Tilldone, both writes would have happened inside a single declared task and the duplication would have been invisible. With Tilldone, the first write had to close the task before the second write could start. The second write would have to begin a new task, and beginning a new task forces a re-declaration of intent. The agent, looking at its own state, decided the duplicate write was unnecessary and skipped it.
Example 2: the verifier that kept agreeing. A review agent was getting a verifier yield that said "all checks passed" on a test that was not, in fact, passing. The verifier was a small model with a high false-negative rate. The Damage Control classifier saw three successive "all clear" yields on a stage the upstream chain had explicitly flagged as failing, classified them as correctness failures (not transient), escalated, and halted the dispatch. The human in the loop was paged with the receipt. The verifier was swapped out for a stricter one, and the test failed the way it should have.
Example 3: the meta-dispatch that wanted to spend $4. A Purpose Gate call came in for a single-file refactor. The agent had drafted a meta-dispatch that would have spun up six expert agents, burned roughly $4 in API spend, and taken twenty minutes to produce a one-line change. The gate read the declared purpose ("refactor this function for readability"), read the proposed route (six experts), and refused. The task was redirected to a single agent with a $0.10 budget. The refactor completed in forty seconds.
These are the kinds of failures that do not show up in an "AI went wrong" headline. They show up in a security audit three months later, when someone asks why the configuration drift happened, or why the verifier was never questioned, or why a single-line refactor burned a budget line. The primitives turn each of those into a receipt.
Why This Could Be Good
I want to be honest about why this is good news, not just a checklist of controls.
The good news is that the primitives are small. They are not a new framework. They are not a vendor platform. They are roughly 800 lines of Python and Bash, sitting in front of whatever agent substrate you are already running. The total surface area is auditable by a single security lead in an afternoon. There is no SaaS dependency. There is no subscription. There is no "we will get back to you on the SOC 2 question."
The good news is that they are durable. The patterns (completion gates, seat contracts, deterministic classifiers, purpose alignment, read-only experts with a single writer) generalise. They work in LangChain. They work in AutoGen. They work in a raw OpenAI API loop. They work in the local open-weights loop you run on your own workstation. You do not have to adopt pifactory to adopt them.
The good news is that they make the AI agent cheaper to run, not more expensive. Damage Control halts retry storms. Purpose Gate refuses over-scoped dispatches. Meta Pi's read-only experts are smaller models than the writer. The net effect of all five primitives on a typical run is fewer tokens burned, not more. Token-responsibly is not just a security posture. It is a cost posture.
The good news is that the verification story gets easier to tell. When a CIO asks "can we trust this agent?", the answer is no longer "trust the model." It is "trust the receipts." That is a much shorter conversation, and it is one a GRC lead can be present in.
The good news is that this is a tractable problem. The AI agent security conversation in 2026 is dominated by two extremes. There is the vendor pitch that says the model is safe by design. There is the doomer thread that says the model is an attack surface by default. Both readings are incomplete. The reality is that an AI agent is a piece of software with a non-deterministic component, and the same engineering discipline that makes any other piece of software auditable - completion gates, least privilege, deterministic classifiers, receipts - applies here too.
Securing your workplace AI? You are probably your family's first-line IT support too.
If you are the security lead evaluating an AI agent rollout, the same questions apply at home: when did you last check that your family's passwords are unique, that their devices are patched, that their backups actually restore? Token-responsibly at work is the same discipline as password-responsibly at home - spend the budget where independent evidence buys down meaningful risk.
Get my Personal Security Quick-Start Guide - the 193-page practical handbook for busy IT pros who want to protect their families without becoming cybersecurity experts.
Plus: Join 158+ Australians getting one 5-minute security briefing every Friday.
What Could Go Wrong
Three honest concerns.
Concern 1: this is a personal factory, not a vendor product. The primitives are real and the receipts are real, but I am one person running this on a workstation. Your enterprise has a procurement team, a vendor review process, and a SOC 2 checklist. The patterns translate, but you will need to do the integration work. There is no "buy pifactory" button.
Concern 2: a determined adversary will still get through. A completion gate stops the silent failure. It does not stop a prompt-injection attack that convinces the agent the task is legitimate when it is not. A seat contract stops the silent tool expansion. It does not stop a hijacked context that masquerades as an approved seat. The primitives raise the cost of an attack, but they do not make an AI agent immune to one. If you are deploying AI agents in a high-stakes environment, you still need the standard controls - network segmentation, key management, scoped credentials, Defender for Cloud-style posture management - on top of the agent-side discipline.
Concern 3: the open question is what "verified" means. I have a definition: a tool call is verified when the receipt matches the declared purpose, the verifier yield satisfies its atomic claim, and the dispatch cost stayed inside the gate. Your definition may differ. The honest answer is that the security community has not yet converged on a definition of agent completion that a regulator would recognise. Until that exists, "verified" is whatever your team decides it means, and you should write that definition down before you ship the agent.
What You Should Do Right Now
If you are running AI agents in production today, here is the order to act in.
-
Add a completion gate to your highest-risk state-changing tool calls. Take half a day. Pick the top three tool calls in your agent that change state (file writes, database writes, outbound API calls, code execution). Wrap them in a check that asks "is there an active declared task that authorises this call?" If there is not, reject the call with a bounded retry and then halt. Use my Tilldone pattern as a starting point. The gate does not need to be clever. It needs to exist.
-
Write a seat contract for every agent you have running. Take an afternoon. For each agent, list the tools it is allowed to call, the data it is allowed to read, the data it is allowed to write, and the team or context it is allowed to operate in. Reject any call that does not match the contract. This is the principle of least privilege applied to an LLM. Most teams skip this because "the agent can do everything" feels easier. It is not. The cost of skipping it lands in the next audit.
-
Build a per-tool-call receipt stream. Take a day. Every tool call should produce a structured record with at least: timestamp, run ID, agent ID, tool name, declared purpose, purpose alignment flag, token count, duration, cost. The schema is in the pifactory
docs/per-tool-events-contract.md. You do not need to ship the swim-lane visualiser I have not built yet. You need the receipts. The receipts are the audit trail. -
Replace your silent retry loop with a deterministic failure classifier. Take a day. Three buckets. Transient. Correctness. Unrecoverable. Apply explicit rules. Halt on three successive failures of the same stage. Do not let an LLM decide whether to retry. That decision is a security decision, not a model decision.
-
Add a Purpose Gate before any agent dispatch that costs more than a dollar. Take a day. Declare the purpose. Check the route. Check the spend. Refuse over-scoped dispatches. The gate does not need to be sophisticated. It needs to exist and it needs to write a receipt.
-
If you have a meta-agent or a multi-agent setup, split readers from writers. Take a week. Read-only expert agents. One writer/orchestrator. No shared mutable state. Every expert yield is a deterministic handoff. The writer is the only merge boundary. This is the single highest-impact change you can make. It is also the one that requires the most architectural buy-in, which is why it is last on the list.
-
Write down what "verified" means in your environment. Take an hour. If you cannot show a GRC lead the definition of a verified agent run in your environment, you do not have one. The definition is the foundation for every audit question that comes next.
Key Takeaways
- AI agents fail silently, not loudly. The fastest path to detect that is deterministic checks at every tool call, not "more LLM-as-judge." A model that says "done" is not the same thing as a model that is done.
- Token-responsibly is the right instinct for security work. Pay for the smartest model only when the gate requires it. The cheapest run that passes verification is the best run, not the most expensive one.
- Completion gates beat completion prompts. A shell script that checks for an active declared task is harder to bypass than a system prompt that asks the agent to declare its task. Use the shell.
- Seat contracts beat seat permissions. "This agent can call these tools" written down and enforced by a dispatcher is auditable. "The agent is generally well-behaved" written into a prompt is not.
- Receipts are the audit trail. Per-tool-call receipts with purpose alignment, cost and duration are the single highest-impact investment you can make. They make every other control easier to explain.
- Read-only experts, one writer. The closest I have found to least privilege for a multi-agent system is eight read-only experts feeding one writer/orchestrator. The writer is the only merge boundary.
- The primitives are durable. The five patterns generalise to any agent substrate. You do not need to adopt pifactory. You need to adopt the discipline.
FAQ
Q: What is an AI agent "completion gate"? A: A shell-level check that asks, before any state-changing tool call runs, whether there is an active declared task that authorises the call. If there is not, the call is rejected with a bounded retry, then halted. The gate makes "done" a system-level property, not a model claim.
Q: Why deterministic checks instead of LLM-as-judge for verification? A: LLM-as-judge adds tokens, adds latency and adds a new non-deterministic component to the chain. A deterministic check (a shell script reading structured output) is cheaper, faster and auditable. The verifier model can still exist; it just should not be the only thing standing between a silent failure and a shipped run.
Q: What is "token-responsibly"? A: My phrase for spending tokens only where independent evidence buys down meaningful risk. It is the opposite of TokenMaxxing. A Purpose Gate is the operational form of the judgement. If the task is well-scoped, you do not need a meta-agent. If the task is ambiguous, you do.
Q: Do these primitives work with any agent framework? A: Yes. They are patterns, not a framework. They work with LangChain, AutoGen, a raw OpenAI or Anthropic API loop, an open-weights local loop, or a custom harness. The integration cost is a wrapper script per state-changing tool call and a structured receipt stream.
Q: How is this different from a standard audit log? A: A standard audit log records what the agent did. These primitives record what the agent declared, what the system checked, and whether the check passed. The difference is the purpose-alignment field: every receipt carries the declared purpose of the tool call and a flag for whether the call stayed aligned with that purpose. That is the field a security audit cares about most.
Q: Will these primitives stop prompt injection? A: No. They raise the cost of an attack by making every state-changing call conditional on an active declared task. A successful prompt injection can still authorise the call. The primitives do not replace standard controls (network segmentation, scoped credentials, Defender-style posture management). They complement them.
Q: How long does it take to ship the first primitive (Tilldone)?
A: Roughly half a day for a single engineer. A task-state.json schema, a begin/done wrapper pair, and a state-changing tool-call wrapper. The test suite is a handful of cases. The hardest part is identifying every state-changing call in your existing agent, which is also the most useful audit you will do all week.
Q: Where do I start if I only have a day? A: Step 1 in the action list. Add a completion gate to your highest-risk state-changing tool calls. Ship it. Then move to step 2 (seat contracts) the next day. The two together are the foundation for everything else.
My Take
Here is what I actually think, after shipping these primitives.
The first thing is that AI agents fail silently. That is the default failure mode. Every other failure mode - the model hallucinating, the agent looping, the context overflowing, the verifier disagreeing - shows up in a log somewhere. The silent failure shows up in nothing. It is the absence of a signal, which is the hardest thing to detect in any monitoring system. The fastest path to detecting it is not a smarter model. It is a shell script that writes a single number to a file every time a state-changing call is made. That is the cheapest, smallest, most useful primitive I have shipped in 2026.
The second thing is that TokenMaxxing is the wrong instinct for security work. The instinct, when an AI agent fails, is to throw a smarter model at the failure. Add a verifier. Add a meta-verifier. Add a reasoning layer. Add a self-reflection loop. Each layer adds tokens, adds latency and adds another non-deterministic component to the chain. The security instinct should be the opposite: spend tokens only where independent evidence buys down meaningful risk. A verifier that costs $0.001 per call and catches a correctness failure is a bargain. A meta-agent that costs $4 per call to second-guess a one-line refactor is waste. The Principle Gate is the operational form of that judgement. It is not glamorous. It is a script that says "no" when the route is over-scoped. That is what security work looks like in 2026.
The third thing is that this is durable. The post is short. The primitives are small. The patterns generalise to any agent substrate you are already running. The five primitives - completion gates, seat contracts, deterministic failure classifiers, purpose gates, read-only experts with a single writer - are not a vendor platform. They are a discipline. You can adopt the discipline without adopting pifactory. Most of what is interesting here is the engineering choice, not the code.
The fourth thing, which is the thing I keep coming back to: the smartest AI agent in the world is still less trustworthy than a script that wrote a single number to a file. Build around the script.
Further Reading
- Anthropic Project Glasswing: AI Finds 23k Bugs, But Who Patches Them? (SecureInSeconds - the find-vs-fix gap at vendor scale; the same shape of problem at a different speed)
- Your AI Agent Runs as You: A Security Review (SecureInSeconds - the harness as the attack surface; contains the five-rung ladder for containing the shell)
- Microsoft Copilot Rollout is a Security Disaster (SecureInSeconds - the canonical Copilot security review; same audience, vendor-scale)
- Run Frontier AI Locally on an SSD: The Cheap-and-Secure Answer (SecureInSeconds - what changes when the agent runs on your own hardware)
- What Claude Mythos Means for AI Security (SecureInSeconds - the Mythos tier and what it implies for agent capability governance)
- Microsoft MDASH: The Other AI Vulnerability Hunter (SecureInSeconds - Microsoft's competing agent; useful comparator)
- Shadow IT Is a User-Research Signal (SecureInSeconds - the shadow-AI variant of the same problem)
- Microsoft Defender Portal Migration March 2027 (SecureInSeconds - the kind of standard control you still need on top of agent-side discipline)
Mathew Clark Founder, SecureInSeconds Currently: watching a completion gate reject a state-changing call and feeling unreasonably pleased about it
