Skip to main content

Shadow Agents: The AI Feature Legal Found Out About During the Incident Review

· 10 min read
Tian Pan
Software Engineer

The worst place to discover that you shipped an AI agent is in an incident review. Not the design doc, not the architecture review, not the change ticket — the incident review, where a lawyer is asking why a customer's account got a refund nobody authorized, and an engineer is scrolling through a service that was last meaningfully reviewed eleven months ago, and there, forty lines into a function called enrichTicket, is a model call that reads the customer record, decides on a resolution, and calls the billing API. Nobody diagrammed it. Nobody approved it as an agent, because to the person who wrote it, it wasn't one. It was "just a helper."

This is shadow AI, and it has grown teeth. The first wave was employees pasting company data into consumer chatbots — a data-leakage problem, bad but bounded. The second wave is agents: model calls wired into internal tools, reading real data and taking real actions, sitting inside services that were approved for something else entirely. Roughly half of employees admit to using AI tools their employer never sanctioned, and a striking share of that usage comes from the top — directors and executives are among the biggest culprits, not the interns. When the same instinct reaches your codebase, you don't get a leaked spreadsheet. You get an autonomous actor with production credentials that nobody signed off on.

The uncomfortable part is that shadow agents aren't created by reckless people. They're created by good engineers doing exactly what you asked them to do: ship value fast, reuse existing infrastructure, don't file a ticket for every small change. The governance gap isn't a discipline problem. It's a definitional one — your review process doesn't have a category for what they built.

How an Agent Sneaks Into Production Without Anyone Deciding To Ship One

Shadow IT had an obvious shape. Someone signed up for a SaaS tool, expensed it, and IT eventually found the charge or the DNS traffic. There was a moment of decision — a signup form, a credit card — that left a trail. Shadow agents rarely have that moment. They accrete.

It usually starts inside a service that already exists and already passed review. A support-triage worker, say, that classifies incoming tickets. Six months ago someone added a model call to summarize the ticket body — helpful, low-risk, nobody blinked. Then a second call to suggest a category. Then, because the suggestion was good, a change to auto-apply the category when confidence was high. Then a small tool: if the ticket is a known billing error under fifty dollars, issue the credit automatically. Each step was a one-line diff to an approved service. No single change looked like "we are now deploying an autonomous agent that moves money." But that is what the sum of them is.

The other common path is the vendor toggle. A SaaS tool you already pay for ships an "AI assistant" or an "agentic workflow" feature, and someone enables it because it demoed well. Now a third party's agent is reading your data and calling your tools, and the decision that put it there was a checkbox in an admin panel, not a procurement review. Analysts expect task-specific agents to jump from under 5% of enterprise applications to roughly 40% by the end of 2026 — most of that expansion will arrive as features inside software you already run, not as systems you consciously chose to build.

Both paths share a property that makes them dangerous: there is no artifact that says "an agent now exists here." The org chart of your software has a new autonomous node, and it was added by a diff or a checkbox that no reviewer recognized as consequential.

Why the Controls You Already Have Don't Catch Them

You probably have real controls. Code review, change management, access audits, a security team that scans for unsanctioned SaaS. The problem is that every one of those controls was designed around a human-shaped adversary or a human-shaped operator, and a shadow agent fits none of those templates.

Start with identity. Your audit trails are built to answer "which person did this." An agent authenticates as a service account or an API key — a non-human identity — and in most enterprises those already outnumber human identities by a wide margin and are governed far more loosely. When the agent issues a refund, the log says the billing service did it, which is technically true and completely useless for accountability. There is no login that looks human, no session that a "suspicious sign-in" rule will flag, no name to put in the postmortem's owner field.

Then network detection. The classic shadow-IT playbook watches for traffic to unapproved domains. A shadow agent living inside an approved microservice generates no such signal — its model calls and tool calls look like the normal internal chatter of a service you already blessed. Shadow AI hides inside already-approved applications, invisible to the domain blocklists and app inventories that caught the last generation of rogue tools. The difference between shadow IT and shadow AI is often described as speed — shadow IT sprawled infrastructure, shadow AI accelerates data exposure — but for agents specifically it's also shape: the risky thing is an action taken inside a system you trust, not a connection to a system you don't.

Finally, code review itself. Reviewers are trained to catch bugs, security holes, and style violations in the diff in front of them. They are not trained to notice that a three-line addition just crossed a governance boundary — that this particular model call now reads regulated customer data, or that this tool invocation now performs an irreversible action. The diff looks fine. It compiles, it's tested, it does what the PR says. The reviewer has no prompt that says "wait — did we just build an agent?" And so the most consequential architectural change in the service sails through as routine.

Loading…
References:Let's stay in touch and Follow me for more thoughts and updates