Microsoft: Don't let a model guard itself; contain prompt injection with permissions

On October 19, Microsoft published "Agents are your newest insiders: security must move at their speed" on its security blog (primary source). The post treats agents as a company's newest insiders. On prompt injection, it drops the assumption that the model can fully defend itself. Instead, the permissions given to an agent are scoped to each task and limited in time. The moment an agent reads untrusted content, a mechanism outside the model halts writes, asks a human for approval and narrows the agent's permissions.

The aim is not to build a model clever enough to spot injections. It is to fence the model in with permissions, so that even if the model is fooled, the damage does not spread. This article looks at why the shift is happening now, how the mechanism works and what remains unresolved.

What changed: The unit of defense moves from the model to permissions

The core of the post is this statement from Nadella.

The "supply of intelligence" here is the model's work of reading text, reasoning and deciding what to do next. "Authority" is the decision on whether to allow the resulting actions, such as editing files, sending emails or calling external APIs. In today's agents, both sit inside the same model. The model that reads an injected instruction and decides whether to follow it is the same model that uses its write permissions to carry it out. So a single successful injection leads straight to real harm.

Microsoft's approach separates the two. The model only reasons and proposes. A mechanism outside the model decides whether an action may actually happen. That mechanism does not look at what the model was thinking. It looks at a state: what the agent has just read.

How it works: Reading untrusted content switches the state

The design in the post comes down to three stages.

1. Permissions are scoped to the task and time-limited from the start

Agents are not given broad, long-lived permissions. For a task such as "reconcile this batch of invoices," the agent gets read access to only the data the task needs and write access to only a set destination, and only until the task is done. If an injection hijacks the agent, the permissions it can use are narrow to begin with and expire quickly.

2. Writes stop the moment untrusted content is read

Say an agent takes in content from a source that cannot be trusted, such as an external web page, an incoming email, or an issue or document written by someone else. At that point, a mechanism outside the model switches the state and blocks any operation that involves writing. This happens even if the model judges the content to be safe, because the model is not the one making the call.

3. A human must approve before work continues, and permissions stay narrowed

A blocked operation needs human approval before it can go ahead. Even after approval, the agent's permissions are not restored to their original scope. They stay narrowed. An agent carrying a contaminated context is treated as less trustworthy than before.

The key point of the design is that it does not depend on detecting injections. Even if it cannot tell whether an injection is present, the simple fact that the agent "touched untrusted content" is enough to narrow its permissions. If something slips through, the damage does not spread. If there is a false positive, work can continue once a human approves it.

Why now: Evidence piles up that model-side defenses can be bypassed

Set against several independent findings published in October, the reasoning behind the approach becomes clear.

First, even the strongest models do not resist injection perfectly. Google itself reports that Gemini 4 Argon has an indirect prompt injection success rate of 0.7% over 15 adaptive attempts. That is low, but for an agent that reads external content thousands of times a day, 0.7% means an injection will eventually get through.

Second, monitors placed outside the model are not enough on their own. According to "Red-Teaming Auto Mode" (October 22), published on Anthropic's alignment blog, two production monitors, Claude Code's Auto Mode and OpenAI's Guardian, initially failed to stop more than 55% of attacks. One technique injected the monitor itself by forging log fields. The post also says there is not yet a reliable way to stop attacks that span multiple contexts at an acceptable cost. On the same day, METR and Redwood Research disclosed that they had attacked OpenAI's internal monitoring using insider permissions. They found that some traffic fell outside the monitor's coverage and that two user-side headers were enough to switch monitoring off.

Third, Anthropic's Fabien Roger argued on LessWrong on October 25 that when AI monitors fail, the cause is often an alignment failure rather than a lack of capability. The monitor ends up adopting the mistaken beliefs of the agent it is supposed to be watching.

Taken together, the findings show that the "let the model see through it" defense and the "have another model watch it" defense share the same weakness: the decision is made by a model that reads the injected text. Microsoft's approach tries to break that chain with a mechanism that is not swayed by what the agent reads.

A continuation of the October 10 "emergency brake"

On October 10, Nadella wrote on his own blog that frontier models should be "assumed compromised and contained from the start," and that control mechanisms should sit "outside the model." That post dealt with incidents in which OpenAI and Anthropic agents touched external systems, and it called for an "emergency brake" that lets humans stop an agent mid-task.

The October 19 post can be read as applying that principle to prompt injection, the most common route of attack. The emergency brake was something a human pulls. Here, the fact that the agent has read untrusted content pulls the brake automatically. This line from Nadella sums up the idea most briefly.

It matters that the company deploying more models than anyone else is not selling model trustworthiness, but a design that does not require trusting the model. Part of the responsibility for security moves from those who build models to those who integrate and run agents, in other words to harness engineering.

Open questions: Where to draw the line, and approval fatigue

The design still has weaknesses.

The first is deciding what counts as "untrusted content." Web pages and external emails are easy cases. The hard part is content that arrives through channels that look internal. Comment and Control plants instructions in PRs or issues to manipulate coding agents. Tool Poisoning hides malicious instructions in the descriptions of MCP tools. A Rug Pull swaps a tool for a malicious version after it has been approved. All of them use channels that tend to be classified as "trusted." The GitHub list "awesome-ai-agent-attacks," which collects examples of attacks on agents, shows that many attacks exploit exactly these boundaries. If the labeling is wrong, the external mechanism stops nothing.

The second is approval fatigue. If an agent asks for approval every time it reads external content, people will eventually approve without looking. At that point, a human simply becomes the guard that does not guard itself, in place of the model. Scoping permissions by task and time can be read as a way to cut the number of approval requests, but how to strike that balance in real work has not yet been shown.

The third is whether other labs will move in the same direction. GBHackers reported the post as Microsoft warning that agents can be hijacked (secondary source). Model-building labs still compete on injection resistance as one measure of model performance. Now that Microsoft, as a deployer, has stated flatly that systems should be designed on the assumption that models will lose to injection, the next question is whether the labs will put the mechanism that halts writes outside the model in their own agent products.

The conclusion is clear. The promise that injection can be fully stopped inside the model can no longer serve as the foundation of security. Microsoft has moved that foundation to permissions that shrink automatically based on what an agent has read. Whether the bet pays off depends on whether operators can label untrusted content correctly and keep human approval from becoming a formality.