Conventional pen tests check syntax.
AI pen tests check semantics.
Most pen-test firms still test AI applications as if they were web applications. That misses the entire attack surface modern AI introduces — the layer where meaning is interpreted and where autonomous agents take action on the world.
The shift is linguistic.
Software has rules. AI has intent. The two demand different threat models.
Does the code parse? Are inputs sanitized? Is auth enforced at every route? Are dependencies up to date? Does the API return what the docs say it does?
Does the AI understand the user’s intent — or can an adversary make it understand something else? Can meaning be hijacked through context, tone, instruction order, or retrieved content the AI didn’t know it shouldn’t trust?
Ten layers. Ten attack surfaces.
ZIVIS’s mental model for thinking about modern AI applications — the same model behind our open protocol work currently under IETF review. Each layer has a distinct threat model. A pen test that only covers the bottom two is missing most of the surface.
Can you reconstruct what happened across systems? Logs, traces, audit trails, and the evidence you'll hand a reviewer six months from now.
Policy, approvals, escalation, human review. The mechanisms that decide which actions need a human in the loop — and what happens when they're skipped.
Who is the AI allowed to act as? Identity, OAuth scopes, capability tokens, and the trust boundary between the AI and the systems it touches.
What can the AI actually do in the world? Tools, APIs, side effects, multi-step plans, multi-agent collaboration.
Agent loops, orchestration, retries, routing, tool calls. The plumbing that turns a model output into an action — and the place where one bad signal can cascade.
What does the AI carry across turns and sessions? Conversation state, vector stores, learned preferences, persistent context.
Interpretation, intent, semantic manipulation. Where attackers tunnel through meaning to make the system understand something it shouldn't.
Model inference, decision boundaries, planning. The logic the model applies to context — and where it can be steered into the wrong conclusion.
Prompts, RAG, tool descriptions, input data. Everything the model treats as ground truth before it reasons.
Models, datasets, embeddings, evals, dependencies. The foundation everything else inherits — if it's compromised, every layer above is, too.
Read ATPS bottom-up. Each layer assumes the integrity of the ones beneath it. Compromise the supply chain — and the model lies before you ask. Compromise meaning — and the system acts on the wrong intent. Compromise governance — and observability becomes theater. An attacker only needs to compromise one layer.
Coverage your reviewer will actually recognize.
We test every category in OWASP’s LLM Top 10. We go deeper into the Agentic AI Top 10 — because that’s where the modern attack surface is actually growing.
Engineers who build the exploit. Not consultants who write the report.
Most pen-test firms ship a PDF that lists vulnerabilities. We ship working exploit code that proves what an attacker could actually do.
Book 30 minutes with Jim and Jake
One CISO with 30+ years across enterprise security. One offensive engineer with 25 years finding what scanners miss. One conversation about the deal at risk.
