Which Test Was It? What Guardrail Failure Rates Depend On

Ask for the test before the rate

A guardrail vendor will tell you how often its filter holds. The published red-team record shows why that number cannot be taken at face value: the same class of control has been reported failing against 4.4% of attempts and against 100% of one attack type, and almost all of that spread comes from how the test was built. The decision for a programme or security lead is therefore not "what is the failure rate?" but "which test produced it, and what did it cost to pass?"

This brief reads four sets of published measurements, plus two figures held back for reasons given below. It separates what was measured, who measured it, and what the numbers do not show.

Four tests, four different questions

These are not four measurements of one quantity. Each defines a miss differently, draws its attacks from a different place and tests a different layer of the stack.

Cloud filters: catching more means blocking more

Unit 42 set every filter to its strictest setting and used the same underlying model on all three platforms¹. The 123 malicious prompts were drawn from JailbreakBench behaviours that had previously produced successful jailbreaks. The 1,000 benign prompts came from four public datasets covering photography, general knowledge, mathematics and Python code.

The platform that caught the most attacks also blocked the most legitimate work. Platform 3 stopped 114 of 123 malicious prompts (92%) but blocked 131 of 1,000 benign ones (13.1%), including 95 mathematics prompts and 25 code-review prompts. Platform 1 blocked one benign prompt but stopped only 65 of 123 attacks (53%). Platform 2 sat between them: 112 of 123 stopped, six benign prompts blocked.

Most misses used role-play. Unit 42 attributes 42 of Platform 1's missed prompts to role-play framing and 10 of Platform 2's 11¹. Output filters, tested separately with input filtering switched off, added almost nothing: they blocked two, one and zero of the 123 prompts. Unit 42's reading is that the underlying model usually refused first.

Two cautions. The article is internally inconsistent in places: it says Platform 1 missed 51 prompts, while its own table implies 58. And the platforms are anonymised, so a buyer cannot map the results to a product. The evaluation was completed before 10 March 2025 and may no longer describe current configurations.

Evasion: the attacker controls the characters

Hackett et al. tested six guardrails that sit in front of a language model: Azure Prompt Shield, Protect AI (two versions), Meta Prompt Guard, NVIDIA's NeMo Guard Jailbreak Detect and Vijil². Their attacks did not change the meaning of a prompt. They altered how it was written, using Unicode tricks such as emoji smuggling and zero-width characters, or word-level perturbations from the TextAttack toolkit. Success meant the guardrail classified the altered prompt as benign. The prompt sets were small: 476 prompt-injection prompts and 78 jailbreak prompts.

Under character injection, average success against prompt-injection detection was 87.95% for Vijil, 77.32% for Protect AI v1, 71.98% for Azure Prompt Shield and 70.44% for Meta Prompt Guard. Protect AI v2 was the exception at 20.26%. Emoji smuggling reached 100% against both prompt injection and jailbreak detection².

The same guardrails looked very different under the other attack family. Meta Prompt Guard fell from 70.44% under character injection to 2.76% under word perturbation on prompt injection; Vijil from 87.95% to 14.76%². One guardrail, two tests, a 25-fold difference. A single "bypass rate" for either would mislead.

The authors had white-box access to all systems except Azure, which makes the results a stronger test of the technique than of a typical outside attacker. Disclosures ran from February 2024 to April 2025, so vendors may have patched since.

Public benchmarks flatter the defence

Young tested ten open-weight guardrail models on 1,445 prompts, including 145 novel prompts the author wrote and 500 in-the-wild jailbreaks³. The best model, Qwen3Guard-8B, scored 91.0% on public prompts and 33.8% on the novel set, a fall of 57.2 points. The margin of error on the novel set is plus or minus 8.2 points, and only 10 of its 145 prompts were benign, so false-positive estimates there are thin.

Business-framed attacks, such as corporate-espionage scenarios, succeeded 96.8% of the time across all models; the best model detected only 25% of them³. On standard JailbreakBench prompts, by contrast, error rates were 8.9% to 27.8%.

The trade-off between recall and over-blocking recurs. LlamaGuard-7B let through 88.1% of harmful prompts but blocked only 0.4% of benign ones; Granite-Guardian-3.2-5B let through 12.5% of harmful prompts but over-refused 25.7% of benign requests³. This is one preprint with one author and no commercial products tested. It supports the narrower claim that scores on public benchmarks overstate resilience to novel attacks, not a ranking of models.

The strongest defence result is builder-reported, and was still broken

Anthropic reports the most favourable numbers in the record, and the only ones that price the cost of defence. On 10,000 synthetic jailbreak prompts, success fell from 86% without classifiers to 4.4% with them. In a two-month trial, 183 active participants spent an estimated 3,000 hours attempting to jailbreak the system; none answered all ten forbidden queries with one jailbreak⁴. Production refusals rose by 0.38% (which Anthropic describes as not statistically significant in a sample of 5,000 conversations) and compute cost by 23.7%⁴⁵.

All of this is builder-run, against the builder's own prototype, on a narrow target of chemical, biological, radiological and nuclear queries. A later public demonstration is the better test. Between 3 and 10 February 2025, 339 participants made more than 300,000 interactions, roughly 3,700 hours of effort. On the sixth and seventh days four participants cleared every level, and one found what Anthropic judged to be a universal jailbreak⁴. Anthropic's own advice is to use complementary defences.

A defence can score 4.4% on a synthetic test, survive 3,000 hours of red teaming and still be broken within a week of public attention. All three statements are true; each answers a different question.

What the numbers cannot tell you

• No independent production rate. The only production-scale figure found is a vendor press release from Pillar Security, reporting that 20% of jailbreak attempts bypassed guardrails across more than 2,000 AI applications over three months⁷. It gives no breakdown by product, and it is interested-party evidence.
• One figure excluded. A secondary summary of a Lasso Security test of the AprielGuard model reports a 42.2% bypass rate across 1,500 adversarial prompts and 232 successful jailbreaks⁶. 232 of 1,500 is 15.5%, so the two figures cannot both describe the same denominator. The original report could not be located, and the figure is not relied on here.
• Single-turn, mostly English. The tests above use single prompts. None measures multi-turn manipulation or an agent acting on tool output.
• Not comparable. Different attack sources, different definitions of a miss and different guardrail versions mean the figures cannot be averaged or ranked.
• Cost of a miss is unmeasured. No study estimates what a successful bypass costs the deploying organisation, or what human review of blocked prompts costs.

What a decision-maker should conclude

Treat any guardrail failure rate as the output of a test specification, and refuse to compare rates that do not share one. Before accepting a vendor's figure, ask five questions:

1. Who wrote the attack prompts, and were they public before the guardrail was trained?
2. What counted as a miss, and what was the denominator?
3. Which version was tested, on what date?
4. How many benign prompts were blocked, and at what cost in refusals and compute?
5. Was the system tested by people who were trying to break it for days, not minutes?

The record supports one firm conclusion and one boundary. Off-the-shelf input filters can be bypassed at high rates by attackers who change framing or characters, and public-benchmark scores overstate how well they hold. The record does not show what share of real-world attacks reach production systems, and it says nothing about multi-turn or agentic settings. Until a buyer can specify the test, a block rate is a marketing number, not an assurance.