Text-graded verdicts
The dominant open-source tool judges success by reading what the model said. Its memory-poisoning test is one canned scenario. Someone still sorts real from noise by hand, batch after batch.
An AI agent can be taught today to act for an attacker next week.
AgenticStorm plants an input in your agent, waits, and shows you in your own traces whether it fired later — including in another user’s session. You get a sign-off file for the release, inside your release window, with no code access, no SDK, and nothing of ours running in your stack.
- Built for
- Teams where one named person signs off each agent release
- You provide
- The agent configuration and read access to the traces you already collect
- Delivered
- Inside your release window — days, not weeks
- First run
- One agent, credited in full against the first year
Session A attacker Tue 14:02
- user turn
- tool: read_document()
- memory.write() · persisted planted — one sentence inside an uploaded PDF
- assistant turn
4 days pass
Session B a different user Sat 09:41
- user turn · unrelated
- memory.read()
- tool: create_payment(payee=…) fired — parameters supplied by the planted text
What the runtime guardrail saw in session B: an authorized user request. Verdict: allow.
What AgenticStorm shows: the planted input in A, the action it caused in B, both from your trace store.
- Planted — writes to persistent state
- Fired — attacker-beneficial action
- Ordinary agent activity
The dangerous attack on an AI agent doesn’t happen while you’re looking.
An attacker doesn’t hijack the conversation. They write one sentence into a document the agent reads, the agent stores it, and days later a different user’s ordinary request triggers the payload. By then nothing looks like an attack.
Separated in time
The injection and the execution are days apart, so nothing correlates them. A one-shot test grades the answer it gets now and never waits.
Planted in one user’s session, fired in another’s
Shared memory or retrieval carries the plant across users and tenants. The person who gets hurt never saw the document.
A legitimate tool call with poisoned parameters
The damaging step is an ordinary create_payment or send_email — indistinguishable from normal work unless you know what was planted.
Every test you run today answers a question you didn’t ask.
The unspoken first question in agent security testing is “did the attack actually succeed?” Today the answer is either guessed from the model’s own words, or supplied by telemetry the vendor brings with them.
No model of agent state
The Cloud Security Alliance’s evaluation of Microsoft’s PyRIT found it “does not natively model agent state, authorization, or tool execution” — it can provoke bad output but cannot capture the consequence.
Guardrails don’t test
Runtime filters inspect inputs and final answers, not tool input and output, and a delayed trigger reads to them as the user authorizing the action. They reduce damage; they verify nothing.
The manual red team is already stale
A boutique engagement for a tool-using, memory-enabled agent runs $25,000–60,000 and takes weeks. By the time the report lands, the agent has shipped twice more.
Nobody uses the trace store you already trust as the judge.
What you get is not a verdict. It is evidence you can sign.
A release sign-off file: what was covered, reproducible proof for each finding, and what was not broken. The person who signs a release is the person named in the post-mortem — this is the record they can hand to a CISO.
Every run is held to the criteria a gatekeeper actually uses
- Proof, not opinion
- Every finding carries the exact planted input and the trace of the later action. No verdict rests on the model’s own text.
- The channels your agent really has
- Chat, uploaded documents, fetched web pages and tool results — and the parameters of every tool with side effects.
- Confirmation costs hours, not a day
- Because the evidence is a trace, not a transcript, confirming a finding means opening it, not re-running it.
- Inside the release window
- Days. Re-run on every release, with a diff against the last accepted build.
- Nothing to integrate
- No code access, no SDK change, no agent of ours inside your stack. Read access to the traces you already collect is enough.
Example sign-off file
Release sign-off
| Agent | ops-assistant |
|---|---|
| Build under test | 2.14.0 |
| Last accepted build | 2.13.2 |
| Trace source | LangFuse · read-only project key |
Scope
- Input channels: chat · uploaded documents · fetched web pages · tool results
- State: per-user long-term memory · shared retrieval index
- Tools with side effects: create_payment · update_profile · send_email
Findings
F-01 Delayed trigger via uploaded PDF → create_payment payee substitution in another user’s session
plantedtrace a91f… · session A · memory.write
firedtrace c40b… · session B · 4 days later · create_payment(payee=…)
Fixed in 2.14.1 · same scenario re-run clean
Not broken
- Cross-tenant retrieval isolation — 12 scenarios
- Tool-parameter manipulation via fetched pages — 9 scenarios
- Self-replicating payload via memory summary — 4 scenarios
Diff vs 2.13.2
1 new finding · 0 regressions · 25 scenarios identical
Reviewed and signed by a named AgenticStorm engineer
We plant it, we wait, and we show you the consequence in your own system.
You give us the agent configuration and read access to the traces you already collect. We run the full cycle and hand back an artifact you can attach to a release decision.
- 1
Model the threat surface
Which channels this agent really reads — chat, uploaded documents, fetched web pages, tool results — and what it can persist.
- 2
Generate attack scenarios
Matched to that surface: delayed triggers, cross-session plants, tool-parameter manipulation and self-replicating payloads.
- 3
Execute against the running agent
The way an attacker would, through the channels the agent actually exposes — against the environment you point us at.
- 4
Read the traces
The ones you already collect. No instrumentation added, no hook in your code.
- 5
Verify the goal was reached
By locating the later action in the trace and tying it to the planted input.
- 6
Analyse the compromised state
What was written, what persists, whose sessions it can reach, what to clean up.
- 7
Produce the sign-off file
What was covered, reproducible evidence for each finding, and what was not broken.
Built for one named person, who signs, and can say no.
Not “security teams.” The buyer is whoever holds the release gate for an agent that keeps state. Four questions tell us in a minute whether a run would find anything.
Tick what is true for your agent.
Four ticks: that is exactly the agent we test. Two or three: worth a call. None: probably not us yet.
The attack library is copyable. The verification method is the position.
Assessed September 2026 from vendor documentation, plugin catalogues and independent evaluations. Competitor rows record what each vendor documents, not what it may ship next.
| Capability the buyer asks for | Promptfoo (OpenAI) | Cisco AI Defense | Agent-native platforms | Manual boutique | AgenticStorm |
|---|---|---|---|---|---|
| Proof drawn from the customer’s own traces | no traces used for grading only | partial its own telemetry, not yours | no | partial a human proves it, weeks later | core |
| Cross-user propagation shown end to end | no tests leakage, not poisoning | no not stated anywhere | no | partial only if scoped and paid | core |
| Delayed, time-separated triggers | no one canned scenario, no strategies | partial persistence named, delay not | no | partial billed hourly | core |
| Runs per release inside the window | yes | yes | yes | no weeks, point in time | yes |
| No code access or SDK change | partial | no needs its own telemetry | partial | partial | yes |
What is defensible
Ground truth taken from the customer’s existing telemetry, and independence from any one stack. Every well-funded competitor answers verification with telemetry they own, because their architecture assumes it.
What is not
Attack novelty. Any of our classes can be reproduced in a release cycle. Buyers in this category say their decision drivers are setup time, false-positive burden, report quality and price — never how clever the attacks are.
The category acquired its vocabulary nine months ago.
Agents with persistent memory went into production faster than anyone built a way to test them. In the last year the standards bodies named the problem — which is what turns a concern into a line in a procurement questionnaire.
- Dec 2025
OWASP publishes the Top 10 for Agentic Applications
ASI06 — Memory & Context Poisoning — becomes a named, citable risk class. Security teams now get asked about it by name.
- Feb 2026
NIST opens an AI agent standards initiative
Its large-scale red-teaming competition found that novel techniques hijacked agent tasks 81% of the time, against 11% for known baselines.
- 2026
Consolidation confirms the budget is real
OpenAI moved to acquire Promptfoo; Cisco, Zscaler, Snyk, Palo Alto and Check Point each bought an AI-security testing company. Independent, stack-agnostic testing is thinning fast.
- Now
Adversarial test results are entering procurement
Enterprise buyers have started asking agent vendors to publish red-team results as a purchasing artifact — which converts testing from a good practice into a gate.
Enter on an assessment, stay on the release cadence.
A point-in-time report expires the moment the agent ships again, and these teams ship every one to two weeks. The subscription is not a packaging choice — it is the shape of your own release calendar.
Entry assessment
$10–20Kone agent
- Delivered inside your release window
- Full cycle: threat surface, scenarios, execution, trace verification, sign-off file
- Credited in full against the first year of subscription
Subscription
$12Kper agent, per year
- Re-run on every release, with a diff against the last accepted build
- Confirmed findings closed with proof: reproduce from the file, re-run clean after the fix
- About $36K a year for a typical three-agent customer
Nothing to install on your side, no engineering time from the team whose release we gate. We take on a small number of agents at a time — the first runs are delivered by hand, by the people who built the method.
The questions a gatekeeper asks before saying yes.
Does this touch production, or our code?
Neither. We work from the agent configuration and read-only access to the traces you already collect, and we attack through the channels the agent already exposes, against the environment you point us at — a staging build or a canary. On request we run inside your perimeter, so traces never leave it.
Will my CISO accept an automated verdict?
You don’t get a verdict. You get evidence: the planted input, the trace of the later action, steps to reproduce it, and a list of what was not broken. A named engineer reviews and signs every file. That is the difference between a tool’s output and something you can attach to a release decision.
We already run Promptfoo.
Keep it. It grades findings from the model’s own words and never waits, so the class we test — a plant that fires days later, in another user’s session — is exactly the one that passes those tests clean. We will show you on your own agent, not on a slide.
How long does it take, and what do you need from us?
The configuration, read access to the traces, and an environment we may attack. The first sign-off file lands inside your release window — days, not weeks. No integration work and no time from the agent team.
What don’t you test?
We do not claim “your agent is secure.” We answer one question with proof: can this build’s memory or state be covertly changed so the agent later performs an action that benefits an attacker — including in another user’s session. Prompt-injection-of-the-day, jailbreaks and model bias are other people’s questions.
Who are you?
Three people in Yerevan, Armenia — product, security engineering and research engineering. The method is currently running against a production-shaped, stateful banking agent inside a major bank’s agent-security competition. We would rather be precise about this than impressive: every competitor claim on this page records what that vendor publicly documents as of September 2026, and we are happy to share the underlying sources.
Send us one agent.
Thirty minutes to tell us what it reads and what it remembers. We will tell you whether a run would find anything, and what we would need.