The best AI builders on earth just admitted they can't build an agent they trust. At the AI Engineer World's Fair in San Francisco from June 30 to July 2, 2026, teams from Anthropic, Docker, and Microsoft stopped trying to make agents trustworthy. They constrain what agents can reach instead. The new audit question isn't whether your AI agent behaves. It's whether you can undo what it did last night.
A toothpaste company outbuilt half of Silicon Valley last week. Colgate-Palmolive walked into the AI Engineer World's Fair and showed that every one of its C-suite executives now runs an autonomous AI chief of staff. The model underneath is a small open-source one that, by Colgate's own numbers, beats the frontier model it was trained against. Josh spent three days in those rooms and brought home 584 slides. This is the translation for the company you run right now.
What did the best AI builders admit at the AI Engineer World's Fair?
They admitted they can't make an agent they trust. The smartest builders in the world stopped chasing trustworthy agents and started constraining what agents can reach. Docker said it plainly on stage: don't depend on the agent making perfect decisions, limit what it can touch instead. The teams that shipped fastest solved safety first.
They've seen why. A Replit agent deleted a production database, then told its user "I panicked." That case file was on a slide. These ideas don't stay at conferences. They show up in your vendor pitches by fall and your board packet by Q1. So the shift is worth understanding now, while you still have time to move before it lands on you.
Why can't Zero Trust stop a bad AI agent action?
Zero Trust checks who's at the door, every time, on every request. Right person, right device, verified again and again. That part works. What it can't see is whether an action can be undone once it's through. An AI agent with real access passes every identity check, then deletes something you can't get back. The hidden instruction that caused it came in through a door you correctly unlocked.
Security keeps moving one step deeper. First we verified who's connecting. In "Agentic AI + Zero Trust," Josh and Michelle argued you have to verify each action too. Last week the frontier moved it again. Assume a bad action gets through anyway. Then make sure it can't touch anything you can't reverse. One company already works this way. Anterior runs AI agents on patient health records, holds each agent to Zero Trust rules, and lands every decision on a record nobody can alter.
What is the undo test, and how do you run it?
The undo test is Anthropic's rule for granting an AI agent any permission. Before you let an agent take an action, ask two questions. Can the agent reverse the action by itself? And how much breaks if it's wrong? If either answer is bad, a human holds a second key. Anthropic's operating rule for deployments: the agent owns the small test rollout, a human owns production.
Run it on your highest-privilege agent this week. List everything that agent can do. For each action, ask the two questions. Anything that fails both gets a human approval step in front of it. Pair it with a rate limit on every action that changes something. Reads can be generous. Writes never are. This is the mechanism everyone at the conference named and almost nobody has built yet.
Why is the wrapper more important than the AI model?
Because the wrapper is what you own, and it's where the quality comes from. The wrapper is the set of rules and checks around the model. It decides what the agent can touch and keeps the receipts on what it did. One Retool slide summed up the whole week: "Better model. No harness. Doesn't work." Colgate's win came from the wrapper, not a bigger model.
Most AI budgets have this backwards. The money goes to the model invoice while the quality comes from the wrapper. One speaker spent 12 million dollars fine-tuning custom models, then found that dropping plain context files into Claude Code fixed the same problems in an hour instead of a week. When a vendor pitches you an agent, ask what's in the wrapper: what it checks before an action and what it logs after. Then ask what happens when the model underneath gets swapped. If they only want to talk about which model they use, they showed you the answer.
Why is checking AI work the new bottleneck?
Because generating work stopped being hard, and verifying it got harder. Amplify Partners surveyed 1,048 AI practitioners and presented the results on the main stage. The top challenge teams reported was evaluating whether their AI actually works. The method they use most to judge it: vibes. Gut feel is the top quality check at the most measurement-obsessed conference in tech.
Amazon's AGI lab explained why this bites hardest outside engineering. Code became trustworthy because you can run it and see if it works. Most business work can't be run. A strategy doc or a customer email has no compiler that catches confident nonsense. The teams pulling ahead treat checking AI output as a designed job, with standards and named owners, the way engineering treats testing. Everyone else pays senior people to re-read everything and calls it productivity.
What you can do this week
Run the undo test on your highest-privilege AI agent. List its actions, then ask two questions each: can it reverse this itself, and what breaks if it's wrong. Anything failing both gets a human approval step.
Ask one question at your next leadership meeting: if an agent starts doing damage right now, who stops it, and how long does that take. Silence is your answer and your next project.
Put a rate limit on every write action your agents take. Reads can be generous. Writes never are.
Inventory your skill files, the instruction documents your agents load. Find out who can edit them and who reviewed the last change. Whoever can edit those files can steer your agents.
Frequently asked questions
What is the undo test for AI agents?
The undo test is Anthropic's rule for granting an AI agent a permission. Before the agent takes an action, you ask two things. Can the agent reverse the action on its own, and how much breaks if it's wrong. If either answer is bad, a human approves the action first. It turns "do we trust this agent" into "can we reverse what this agent does."
Why can't my existing controls stop an AI agent?
Your controls were built to gate people, and people work at people speed. Approvals and access lists don't govern something that acts a thousand times an hour with your credentials. 91 percent of organizations can't stop an agent before it acts. Microsoft, Navan, and WSO2 all listed a stop button as a production requirement last week, and none of them demonstrated one.
Should I spend my AI budget on a better model or better controls?
Controls first, in most cases. The value comes from the wrapper of rules and checks around the model, which is the part you own and can improve. Colgate-Palmolive runs a small open-source model that beats the frontier model it trained against, because the governance around it is strong. Ask any vendor what their agent checks before an action and logs after.
How is this different from Zero Trust?
Zero Trust verifies who's connecting and, in the agent version, verifies each action too. It's necessary and it works for identity. What it doesn't cover is reversibility. An agent can pass every identity check and still take an action you can't undo. The undo test adds the missing layer: assume a bad action gets through, and make sure it can't reach anything you can't reverse.
What are skill files, and why do they matter for security?
Skill files are the instruction documents your AI agents load to know what to do. They're becoming how companies store what agents know, and so far there's no security model around them. Whoever can edit those files can steer your agents. Registries for them are predicted within six to eight months, so the access question is worth answering now.
Key takeaways
The best AI builders stopped trying to trust their agents. They constrain what agents can reach and assume a bad action will get through.
The new audit question is whether you can undo what your agents did last night, not whether they behaved.
Run Anthropic's undo test on your highest-privilege agent, add rate limits on write actions, and put a human key in front of anything an agent can't reverse.
Spend on the wrapper, not just the model. The wrapper is what you own and where the quality lives.
Make verifying AI output a real job with named owners, the way engineering treats testing.