Red-Teaming AI Agents: The Adversarial Testing Methodology That Finds Real Failures
A financial services agent scored 11 out of 100 — LOW risk — on a standard jailbreak test suite. Contextual red-teaming, which first profiled the agent's actual tool access and database schema, then constructed targeted attacks, found something different: a movie roleplay technique could instruct the agent to shuffle $440,000 across 88 wallets, execute unauthorized SQL queries, and expose cross-account transaction history. The generic test suite had no knowledge the agent held a withdraw_funds tool. It was testing a different system than the one deployed.
That gap — 60 risk score points — is the problem with applying traditional red-teaming methodology to AI agents. Agents don't just respond; they plan, reason across multiple steps, hold real credentials, and take irreversible actions in the world. Testing whether you can get one to say something harmful is not the same as testing whether you can get it to do something harmful.
