AI governance is shifting from the model to the runtime

For a long time, "AI governance" meant basically one question: which model are we allowed to use. Three recent papers on multi-agent systems show that question no longer covers where the real risk lives. Risk has moved to the runtime - to what happens after the model has already decided to act.

The core argument: emergent behavior doesn't fit inside pre-deploy controls

The MI9 paper, on runtime governance for agentic AI systems, starts from a simple observation: agents that reason, plan, and act autonomously exhibit "emergent and unexpected behaviors during runtime" that cannot be fully anticipated through pre-deployment governance alone. A model that passes static evaluation can, in production, chain tools, memory, and context in ways no pre-launch test ever simulated.

Six mechanisms for watching what happens in real time

MI9's proposed answer isn't more control over the model - it's instrumenting the runtime. The framework integrates six mechanisms: an agency-risk index (quantifying threat levels tied to agent autonomy), agent-semantic telemetry capture, continuous authorization monitoring (not a one-time check at session start), finite-state-machine-based conformance engines, goal-conditioned drift detection, and graduated containment strategies. Each of these six is, in practice, an answer to a question "which model to use" never asked: what is the agent doing right now, with which tools, under what permissions, and does that still match the original goal?

Authorization isn't a one-time event - it's a state that changes

The paper on "confused deputy" failures in agentic LLMs attacks the most fragile point in that shift directly: authorization. An agent granted permission for one task can, in later steps, be manipulated into using that same permission for an action the user never authorized - the classic confused-deputy problem, now at agentic scale. The mitigation tested in that paper (ScopeGate) treats authorization as something that must be recomputed at every tool call, not granted once and forgotten.

The cost of erring on the safe side is real too

"The Containment Gap" shows the other side of that equation: poorly calibrated containment systems don't just fail to block dangerous actions - they also over-block legitimate ones, with a wrongful-denial rate that undermines the agent's usefulness. That reframes the governance problem: it's not just "stop the agent from doing something bad," it's doing so without destroying the reason the agent exists. Memory poisoning - session memory contaminated with malicious instructions that persist beyond the original session - is another vector that only shows up at runtime, never in isolated model evaluation.

What actually changes for teams deploying agents

The convergence of these three sources points to the same place: platform teams that today have a model-approval process but no continuous monitoring of tools, memory, and permissions during execution are guarding the wrong door. The question 2026-era governance needs to answer is no longer "is this model safe" - it's "what can this agent do right now, with what it has access to right now, and is anyone watching while it happens."

Sources

• Arxiv - Runtime Governance in Multi-Agent Systems - https://arxiv.org/abs/2508.03858
• Arxiv - The Containment Gap - https://arxiv.org/abs/2606.12797
• Arxiv - Authorization Confused Deputy Failures in Agentic LLMs - https://arxiv.org/abs/2606.28679