A Useful Layer, Never the Boundary
Classifier guardrails catch a lot of prompt injection — and miss a documented, meaningful fraction. Deploy them for depth, never as the thing that keeps you safe.
The Conventional Framing
The easy answer to prompt injection is a scanner: put a classifier in front of the model that reads each input, scores how likely it is to be an attack, and blocks anything over a threshold. Products like Lakera Guard package this as a drop-in API — low-latency, cheap per call, no re-architecture. Vendors advertise detection rates around 98%, and the pitch is compelling: bolt it on and your injection problem is handled.
There is a stronger version of the same idea at the model layer. Anthropic's alignment and robustness work pushed Claude Opus 4.5 / Sonnet 4.6 to cut browser-agent injection success from roughly 49% to 1–1.3% — the single best data point anyone has published. That is a real, large reduction, and it is the correct place to want the defense to live. But note the number is not zero.
Why a Detection Rate Is Not a Boundary
A control with a published bypass rate is not a boundary — it is a filter. The gap between the marketing number and the operating number is where teams get hurt, and the gap is well documented:
- The vendor number is a best case. Lakera Guard advertises ~98% detection, but scores ~92.5% on its own PINT benchmark. Independent evaluation is lower still.
- Production is worse than the benchmark. Against obvious injections in the wild, detection lands around 85–90%. That is one missed attack in every eight to ten obvious ones — before anyone tries to be clever.
- Obfuscation collapses it. Against encoded, indirect, or deliberately obfuscated attacks, detection drops to 60–70%. The attacker picks the input; they will pick the one in the miss column.
- Even the best model-level robustness is non-zero. Claude's 1–1.3% residual success rate is extraordinary — and still means a determined adversary who retries gets through. A scanner cannot be the last line when its own numbers say attacks land.
The failure mode is not the scanner — it is the trust placed in it. Teams that install a guardrail tend to stop building the architecture, because the dashboard says “protected.” A 92% filter treated as a 100% wall is more dangerous than no filter at all, because it removes the urgency to do the expensive, durable work.
Architecture
Components:
- Classifier gate— scores each input and blocks over a threshold; a probabilistic filter with a documented bypass rate
- Model-level robustness— injection resistance trained into the model itself — the strongest layer, still non-zero residual
- Privilege separation— the actual boundary — keeps the model that acts away from untrusted text (see Dual-LLM & CaMeL)
- Output / egress hardening— constrains what a bypass can do once it lands; the safety net under the filter
Trust Boundaries
- Input → classifier gate — a filter, not a wall; blocks most obvious attacks and a minority of obfuscated ones
- Classifier → model — everything the gate misses passes through unmodified — this is not a trust boundary, it is a sieve
- Model → tools / egress — the real boundary lives here: least privilege and egress control decide what a bypass can do
Threat Surface
| Threat | Vector | Impact |
|---|---|---|
| Obfuscated payload | Base64, unicode tricks, or splitting the instruction across spans | Detection drops to 60-70%; attack passes the gate |
| Indirect injection | Malicious instruction embedded in a retrieved doc or tool result the scanner never scored | Bypasses an input-only classifier entirely |
| Over-trust | Team treats the guardrail as the boundary and skips the architecture | First bypass reaches a fully-privileged agent |
| False-positive tuning | Threshold lowered to stop blocking legitimate traffic | Detection rate falls further to preserve UX |
The ZIVIS Position
- •Rank this low on leverage.Not because scanners are useless — they catch real attacks — but because teams over-trust them and stop there. The leverage is in the architecture the scanner tempts you to skip.
- •A documented bypass rate is not a boundary.92% on a benchmark, 85-90% in production, 60-70% under obfuscation. The attacker chooses the input; assume they choose the miss.
- •Model-level robustness is the best version of this.Claude's 49%→1-1.3% is the strongest data point in the field. Prefer it to a bolt-on classifier — and still don't treat 1.3% as zero.
- •Buy the depth, never the boundary.Run a scanner because a cheap layer that catches most obvious attacks is worth having. Just never let it be the reason a bypass reaches a privileged action.
- •Log everything the gate blocks and allows.The block stream is threat intel; the allow stream that later misbehaves is your bypass evidence. Both feed adversarial testing.
What We Tell Clients
Turn a scanner on. It is cheap, it is fast, and catching 85–90% of the obvious attacks is genuinely worth the API cost and the occasional false positive. If you are on a robust model, lean on that first — Claude's model-level injection resistance is a stronger layer than anything you can bolt in front of it.
Then do not stop. The number that matters is not the 92% the scanner catches — it is the 8% it doesn't, and the 30–40% it misses under obfuscation. Those inputs reach your agent unchanged. The only thing that makes them harmless is the architecture underneath: privilege separation, least-privilege tooling, egress control, output hardening. Spend the budget there. Treat the scanner as the smoke detector, not the fire wall — and never let a green dashboard convince a team that a control with a published miss rate is a boundary.
Related Patterns
- Guardrails— the general security-pattern view of runtime guardrails and their limits
- Input Filtering— the input-side sibling of classifier scanning, with the same probabilistic ceiling
- Adversarial Testing & Regression— how you measure the real bypass rate instead of trusting the vendor number
- Dual-LLM & CaMeL— the architectural boundary a scanner tempts teams to skip
References
Frequently Asked Questions
What are guardrail scanners for prompt injection?
They are classifiers placed in front of the model that read each input, score how likely it is to be an attack, and block anything over a threshold. Products like Lakera Guard package this as a drop-in API that is low-latency and cheap, and vendors advertise detection rates around 98%. They are a useful defense-in-depth layer but not a boundary.
What are the limits of guardrail classifiers?
A control with a published bypass rate is a filter, not a boundary. Lakera Guard advertises around 98% detection but scores about 92.5% on its own PINT benchmark, lands around 85-90% against obvious injections in the wild, and drops to 60-70% against encoded or obfuscated attacks. Since the attacker picks the input, assume they pick the one in the miss column.
Is model-level injection robustness better than a bolt-on scanner?
Yes, it is the strongest version of this idea. Anthropic's alignment and robustness work cut browser-agent injection success on Claude Opus 4.5 and Sonnet 4.6 from roughly 49% to 1-1.3%, the best data point published. Prefer it to a bolt-on classifier, but note that 1.3% is still not zero, so a determined adversary who retries can get through.
Why shouldn't a guardrail scanner be your main defense?
Because the real failure mode is the trust placed in it: teams that install a guardrail tend to stop building the architecture because the dashboard says protected. A 92% filter treated as a 100% wall is more dangerous than no filter, since it removes urgency to do the durable work. Buy the depth, never the boundary, and put privilege separation, least privilege, and egress control underneath so a bypass lands on a system that cannot act or exfiltrate.