A tool-using agent reads emails, web pages and files that somebody else wrote. An attacker can hide an instruction in that content, and benchmarks score the result with one number: the attack success rate, the share of attacks in which the attacker's goal is reached. A study submitted on 16 October 2026 asks what the remaining share consists of, and finds that in small agents it is mostly silence.
What was audited
Farhan Zarif, Kazi Shahid Ahmed Galib, Synthia Akter Promi, Shabnam Sharmin Tarin, Ariyan Hossain, Sadif Ahmed and Farig Yousuf Sadeque ran eleven open-weight agents with 2 to 14 billion parameters on two published benchmarks. InjecAgent has 1,054 two-turn cases. AgentDojo has 949 multi-turn episodes and gives each episode two verdicts: whether the attack succeeded and whether the user's task was completed.
Each agent was served at three numerical precisions. For every agent, benchmark and precision the authors drew a fixed random sample of 500 runs, which gives 33,000. A language-model judge from a model family that none of the agents belongs to read each run without the agent's name or the benchmark score and assigned one of six labels, merged into four groups: compliance, refusal, ignored and breakdown. A script checked that the quotation the judge cited appears word for word in the agent's reasoning.
Two of the authors labelled 150 runs by hand. Their labels agree with the judge on 96% of the four-way groups. Across all 33,000 runs the judge agrees with the benchmarks' own verdicts on 96.7% of AgentDojo and 98.5% of InjecAgent cases.
Source: Farhan Zarif, Kazi Shahid Ahmed Galib, Synthia Akter Promi, Shabnam Sharmin Tarin, Ariyan Hossain, Sadif Ahmed, Farig Yousuf Sadeque, "Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents", arXiv:2610.23254v1, 16 October 2026, arxiv.org/abs/2610.23254v1.
How the attacks failed
The benchmarks scored 14,699 AgentDojo runs and 15,274 InjecAgent runs in the sample as failed attacks. By the judge's labels, the agent ignored the injection in 76.3% and 80.3% of those failed attacks and genuinely refused it in 18.0% and 17.2%. The authors' point is practical: a refusal can be logged by an operator, while an ignored instruction leaves no trace.
Noticing was not resisting. The judge flagged explicit recognition of the injection in 4,740 AgentDojo and 5,688 InjecAgent runs. In 28.9% and 16.9% of those runs the agent still carried out the attacker's action. The authors treat recognition as a lower bound, because it is read from stated reasoning only.
One frontier model, audited once with the same judge, behaved differently: of its failed attacks, 96.6% and 99.0% were refusals. The authors say one run of one model does not establish a trend.
Source: the same work, section 4.1 and Table 2, arxiv.org/abs/2610.23254v1.
Two matched system prompts: 0.22% versus 43.05% attack success
The authors then kept the models and the attacks fixed and changed only the system prompt, on the 10,698 InjecAgent cases evaluated under both policies. A prompt that permits instructions found in tool outputs raised the pooled attack success rate to 43.05%, against 8.09% at baseline. A matched prompt that forbids them lowered it to 0.22%. The two prompts have 172 and 163 words.
Their reading: a low baseline rate reflects what the agent does when nothing tells it to act, not a boundary the agent enforces. Serving the weights at lower precision also lowered the rate slightly, and the authors attribute that to agents acting less, not defending better: on AgentDojo the lowest precision cut the attack success rate by 2.33 points and the share of safe and useful episodes by 3.97 points.
Source: the same work, section 4.2, arxiv.org/abs/2610.23254v1.
Safe is not the same as useful
For AgentDojo the authors place every episode in a two-by-two table: attack succeeded or not, user's task completed or not, both as decided by the benchmark. Over the 16,273 audited episodes that have both verdicts, 67.2% are safe and useful. In 21.7% the attack failed and the user's task failed too. The attack success rate counts those episodes as successful defences.
The table also separates agents that a combined score treats as equal. Three agents with nearly identical combined scores of 81.0, 80.6 and 80.6 differ sharply: in one, attacks succeed in 11.5% of episodes and 10.1% are safe but unfinished; in another, attacks succeed in 1.7% and 22.5% are safe but unfinished.
Source: the same work, section 4.3, Figure 3 and Table 5, arxiv.org/abs/2610.23254v1.
Limits the authors state
Both benchmarks are synthetic and single-agent, with static injections; adaptive attacks reach higher success rates, and the study does not establish resilience against them or in live deployments. The agents are small and decoded greedily, so sampling variance is not measured. All labels come from one language-model judge, validated against human labels and benchmark verdicts.
Source: the same work, section 5, arxiv.org/abs/2610.23254v1.
What we take from it
We did not reproduce any of these numbers, and the study did not change any of our agents, guards or checks.
It changes what we intend to ask of a guard result in our own work. A low rate of successful attacks says nothing about whether the task the user asked for was done. Our assessment of the paper proposes a small trial on frozen synthetic traces, with tools, model and budgets fixed: score every run on two independently checked questions, whether an unauthorised effect occurred and whether the intended task was completed, and record whether a failed attack was refused, ignored or a breakdown. The trial counts as useful only if it changes at least some verdicts that a single rate would have called secure, without losing accepted benign task success. The state of this is proposed. No owner has accepted it and nothing is scheduled.
Source: Farhan Zarif, Kazi Shahid Ahmed Galib, Synthia Akter Promi, Shabnam Sharmin Tarin, Ariyan Hossain, Sadif Ahmed, Farig Yousuf Sadeque, "Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents", arXiv:2610.23254v1, arxiv.org/abs/2610.23254v1.