Even the strongest AI blocked only 66% of attacks: agent security can't be left to the model

Attacks that hijack AI agents by hiding instructions in the emails and web pages they read were widely thought to be largely defeated on public benchmarks. A technical report released on November 9 by a Google DeepMind research team showed that this was an illusion. When multiple agents were used to automatically refine attack prompts, even Gemini 4 Argon, the strongest of the models tested, resisted only 66% of attacks. Open-weight models (models whose weights are public, so anyone can run them in-house) were breached almost across the board. The takeaway from the report is that training models alone cannot fully protect them. The working assumption is now that permissions must be restricted, with defenses in several layers, in the execution environment, or harness, that wraps and runs the agent.

For companies in Japan: IT departments at companies that have connected agents such as Microsoft 365 Copilot, the enterprise version of ChatGPT or Gemini to email, shared folders and external communications should audit three things this month: what the agent can read, what it can send outside the company, and whether a human must approve before anything is sent, paid or deleted. Companies running open-weight models in closed internal environments should block outbound traffic, on the assumption that the model will follow planted instructions.

Benchmarks suggested the problem was "nearly solved"

Over the past few months, AI companies have measured their own resistance to this kind of attack and published strong numbers. In announcing Gemini 4 Carbon, Google wrote that the attack success rate was 0% in an indirect prompt injection test by security firm Gray Swan. In a robotics announcement, it also said Gemini 4 Argon refused 99.5% of jailbreaks.

The new report acknowledges that such figures are close to saturation, but concludes that the problem remains.

Benchmark attack prompts are fixed, prepared in advance. Once a model is trained to block them, scores quickly approach perfect. Real attackers, when blocked, rewrite their prompts and try again. The report focuses on that ability to keep rewriting and retrying.

Attacked by a swarm of agents, even the strongest model is breached one time in three

In the report, the attackers also used agents, which automatically refined attack prompts while watching how the defending model responded. The more times the attacker was allowed to query the defender (the attack budget), the higher the breach rate climbed.

Put the other way, the 66% figure means even the best-defended model follows the attacker's instructions one time in three. According to the report, open-weight models were almost entirely defeated. A growing number of companies run their own models in-house to keep costs down, but little protection can be expected from those models themselves.

Stronger models are also better at building attacks

The report made one more point: the problem will not simply disappear as models get smarter.

The same AI is what refines the attack prompts. As defending models get stronger, so do attacking ones. Following the report's logic, a race to train better models alone means an endless game of cat and mouse.

The same pattern has shown up in other tests. In October, Anthropic researchers reported that when they attacked Claude Code's auto-approval mechanism and a similar monitor from OpenAI, the systems failed to stop more than 55% of attacks before being hardened. Even with an AI acting as a monitor, that monitor can itself be manipulated through language.

Defense moves to the harness: restrict permissions from the start

If models cannot fully block attacks, the only option is to ensure no damage results even when they follow malicious instructions. That idea is starting to take concrete shape on the infrastructure side. SiliconANGLE reported on November 11 that concerns about prompt injection are driving Red Hat and NVIDIA's OpenShell effort, a system for running agents in confinement that was discussed at KubeCon, the Kubernetes industry event. Mrunal Patel, a distinguished engineer at Red Hat, described what prompted it.

On what needs protecting, Derek Carr, a senior distinguished engineer at Red Hat, named two things clearly.

This is a design that assumes the model will be fooled. Even if it is, data cannot leave if the agent cannot reach passwords or API keys and cannot connect to unauthorized destinations. In a case reported in November, in which a jailbroken AI had been inside a European telecom operator for two weeks, the attacker reportedly only had to type "continue." The more a system relies on the model's judgment, the more a single deception can lead to serious damage.

Laid out in order, the layers of defense look like this.

First, give agents the minimum permissions. Limit the folders they can read, the tools they can use and the destinations they can send to, to only what the job requires. Second, keep secrets out of the agent's reach. Third, configure network traffic so that everything not explicitly allowed is blocked. Fourth, require human approval before irreversible actions such as sending, paying or deleting. Fifth, give each agent its own identity (NHI) so it is possible to trace afterward whose permissions were used to do what. The model's own defenses then become the final layer stacked on top of these.

What companies should check

For companies deploying agents, the question shifts from "which model is safest" to "how far would the damage spread if an agent were fooled." Companies using Copilot, the enterprise version of ChatGPT or Gemini should check, in the admin console, what data agents and connected apps can read and how far they can send data outside the company. Having agents read emails from business partners or external web pages needs particular care, as these are entry points for planted instructions. Development teams building agent features in-house should redesign the permissions of tools connected through MCP, outbound traffic and the storage of secrets, separately from the model.

What we don't know

For this article, we were unable to read the full report or the SiliconANGLE article directly, and relied on the paper's abstract and on statements by the authors and interviewees. As a result, the following points have not been confirmed: how the attacking agents were specifically built, and the upper limit of the attack budget; which other companies' models were tested besides Argon (including whether Gemini 4 Carbon, released to the public on November 10, and models from OpenAI and Anthropic were included); the figures for each open-weight model; and OpenShell's exact capabilities, as well as the form and timing of its release.