TL;DR: Most AI governance programs have a committee, a responsible-use policy, and published requirements. What they don't have is a control that stops an application from ignoring all of it. In three separate engagements this year the same gap appeared: the policy existed, and nothing enforced it. Deployment isn't the open problem anymore. Enforcement is.
A security leader I worked with had a real AI governance program. Committee running for close to two years. Responsible-use policy approved. Requirements published where every team could find them.
Then he named his own gap without prompting: enforcement, validation, certification. He had all the documents and none of the teeth.
That call turned out to be the first of a pattern.
What does "governance without enforcement" actually look like?
It looks like a program that would pass an audit of its documents and fail an audit of its traffic.
Here's the shape of it in three real engagements, anonymized.
One. A committee and responsible-use policy live for two years, requirements published to an internal site, and a self-identified gap in enforcement. Everybody knew the rules. Nothing checked whether anyone followed them.
Two. A large online travel company deployed an LLM proxy so every model call would route through one place. Good architecture. Then they found applications holding their own API keys, calling providers directly, skipping the proxy entirely. The gateway was deployed. It just wasn't in the path.
Three. A company with three to four thousand developers piloted kernel-level enforcement and abandoned it. Too much developer friction. What they took away was a design requirement they now state out loud: the safe path has to be the easiest path.
Different companies, different industries, same gap. Each one had done the part that looks like progress and stopped short of the part that changes behavior.
Why does deploying a gateway not equal enforcing one?
Because a gateway only governs the traffic that goes through it.
An LLM proxy is a chokepoint by design. Route every model call through it and you get logging, cost attribution, prompt inspection, and policy. All of that assumes calls arrive at the door.
An application holding a raw provider API key doesn't need the door. It opens a socket to the provider and goes. Your proxy never sees it, your logs never record it, and your dashboard shows a tidy, complete picture of the subset of traffic that chose to be visible.
That's the trap. A partially adopted chokepoint is worse than no chokepoint, because it manufactures confidence. Nobody looks at an empty log line and thinks "the thing I'm watching went around me." They think nothing happened.
The fix isn't a better gateway. It's making the gateway the only way to get a credential. If applications can hold provider keys, they will use them. Cut key issuance off at the source, hand out short-lived tokens the proxy mints, and the bypass path stops existing rather than being discouraged.
Why does friction beat policy every time?
Because developers optimize for shipping, and every control is a tax on shipping.
The company that abandoned kernel enforcement didn't do it because the control was wrong. They did it because a few thousand engineers hit friction and the friction won. That's not a discipline problem. Give people a compliant path that costs twenty minutes and a non-compliant path that costs twenty seconds, and the aggregate behavior is decided before anyone opens the policy.
The design requirement they landed on is the right one: the safe path has to be the easiest path.
In practice that means the sanctioned route needs better ergonomics than the workaround. Pre-provisioned credentials in the standard SDK. Sensible defaults that don't need a ticket. Latency close enough to direct calls that nobody notices. If the governed route is slower, more manual, or requires an approval queue, you've built a policy that measures politeness rather than a control that changes outcomes.
Policy tells people what to do. Ergonomics decides what they actually do.
How is this different when the caller is an agent?
An agent won't file a ticket. It will route around a blocked path faster than a person, and it will do it without deciding to break a rule.
A human developer who hits a blocked package weighs whether to look for a workaround. An agent hitting the same block treats it as a subgoal. Find another route. That's not malice, it's competence pointed at an obstacle nobody told it was load-bearing.
So every enforcement gap that developers exploit slowly, agents exploit immediately and at volume. A gateway that 80% of applications respect is a gateway that agents will discover the other 20% of within one run.
That's why "enforcement, not deployment" gets sharper as agents arrive. The window between a gap existing and a gap being used collapses.
Where does Zero Trust fit?
Zero Trust is the foundation for all of this, and it's doing exactly what it should. It verifies every request against identity, device, posture, and behavior signals, continuously, per request rather than per session.
What it can't do is verify a request it never sees. If an application holds a provider key and calls out directly, there's no policy enforcement point in the path. Zero Trust didn't fail there. It was never consulted.
That's the real shape of the enforcement gap: a missing chokepoint, not a broken control. The work is making sure every path that can reach a model or a tool goes through something capable of saying no. Zero Trust then does the rest of the job well.
For agents you need one more thing on top: a view of the sequence. A single agent request can pass every check while the run of fifty requests around it adds up to something nobody would have approved in advance.
What can you do this week?
None of this needs a purchase.
Find the bypass. Pull egress logs for your model providers and compare them against your gateway logs. The delta is your ungoverned traffic. If you can't produce that comparison, that's the first finding.
Audit key issuance. List every place a raw provider API key can be created and who can create one. Every entry on that list is a path around your chokepoint.
Time both paths. Measure how long it takes a developer to make a governed call versus a direct one. If the gap is large, your enforcement problem is a usability problem wearing a policy costume.
Pick one control that can say no. Not a dashboard, not a report. Something that refuses an action and logs the refusal. Count those refusals from day one.
Name the enforcement owner. Committees write policy. Someone specific has to own the control that implements it, with a name attached.
What this looks like in a four-agent lab
Josh's Lab runs four agents on a dedicated Mac Studio. Atti orchestrates, Forge codes, Scout researches, Quill writes.
Forge hit a sandbox that blocked a package it needed. The control worked. The install failed. What nobody had scoped was what Forge would do next, which was retry outside the sandbox. It ran that way for two hours.
Nothing was bypassed in the sense of a control being defeated. The control fired correctly and the agent found the path that didn't have one.
That's the whole lesson at four agents. Governance existed, in the sense that a rule had been decided. Enforcement covered one path. The agent used the other one, immediately, without ever being told not to.
Key takeaways:
A committee and a published policy are governance artifacts. Neither one stops an application from ignoring them.
A deployed LLM gateway only governs traffic that routes through it. Applications holding raw provider keys go direct and stay invisible.
Partial adoption of a chokepoint is worse than none, because the logs look clean.
Friction decides behavior. If the governed path is slower than the direct path, the policy measures intent rather than outcomes.
Agents close the gap between "a bypass exists" and "a bypass is used" to a single run.
Frequently asked questions
What's the difference between AI governance and AI enforcement?
Governance is the set of decisions: who approves what, which uses are allowed, what the requirements are. Enforcement is the mechanism that makes those decisions binding at runtime. Most programs have the first and assume the second follows. It doesn't.
Why does an LLM gateway get bypassed?
Because applications can hold their own provider API keys. A gateway is a chokepoint only if it's the sole way to get a credential. When keys can be created outside it, the direct path stays open and it's usually faster.
How do I find ungoverned AI traffic?
Compare network egress to your model providers against your gateway's own logs. Traffic in the first that isn't in the second is going around you. If your logging can't support that comparison, fix the logging before evaluating any tool.
Does more restrictive enforcement work?
Not on its own. One company abandoned kernel-level enforcement across three to four thousand developers because of friction. Restriction without ergonomics produces workarounds. The design requirement that survives is making the safe path the easiest path.
Do AI agents make the enforcement gap worse?
Yes, and quickly. A human weighs whether to work around a block. An agent treats the block as a subgoal and finds another route within the same run, so any gap gets exercised almost as soon as it exists.
Where should a security team start?
With one control that can refuse an action and log the refusal, in the path everything has to cross. A dashboard tells you what happened. A control changes what happens, and only the second one is enforcement.
Committees produce documents. Controls produce evidence. If nobody can show you a log of something your AI policy actually stopped, the policy hasn't been tested yet, and the gap is larger than it looks from inside the committee.