120 posts tagged with "security"

A security incident review opens with a question the team cannot answer: how did the agent learn the name of the tool it just called? The audit trail shows a tools/call for a tool whose name does not appear in any tools/list response the harness logged. The MCP server cheerfully accepted the call and executed it. The model, asked in a postmortem to explain where the tool name came from, offers no answer because there is none — it guessed, and the guess landed on a real action.

This is the failure mode at the seam between two assumptions that look compatible on paper. The client treats the tool list as a contract that names the surface area of authority it has been granted. The server treats the tool list as a snapshot of what is currently available, free to grow when the world grows. Between those two views, the LLM is a bridge that does not know the difference.

A new engineer joined the team on Monday. By Wednesday, she had a working local agent setup: an MCP server bridged to the company's deployment API, pointed at staging, talking to her editor. The onboarding doc walked her through the OAuth flow. The token she pasted into the server's environment file was the one her teammate had emailed her — the same token the CI pipeline uses to ship to staging. By Friday, she had joined the team for a working session at a coworking space.

The MCP server was still running. Bound to 127.0.0.1. No authentication. The token was loaded into the process. She didn't think about it because she was not using it. But any tab that visited any website that day could speak to her local server through her own browser. So could any other laptop on the coworking wifi, because she had not noticed that the server was actually bound to 0.0.0.0. The OAuth token your CI pipeline uses to push to staging was now reachable by anyone who could trick a browser into making a request to a local IP — which, in 2026, is a one-pop-up problem.

This post is about that class of failure: the gap between "I'm developing on my laptop" and "my laptop is a server reachable by adversaries." MCP servers, by design, sit right in that gap. Most teams have not noticed.

The design document said each tool gets its own OAuth token, scoped to the minimum permissions that tool needs. The implementation stored tokens keyed by (user_id, provider). Both statements were true on the day v1 shipped, because there was exactly one tool per provider. The day a second tool against the same provider went live, the design document was still true and the storage layer silently invalidated it.

Six months later, a security review traced an incident back to that line of schema. A calendar-reader tool, compromised through a prompt injection in an event description, had successfully called events.delete on the user's primary calendar. The reader had never been granted that scope. The writer had. The token store didn't distinguish between them.

This is the failure mode where a per-provider key shape silently aggregates privilege across tools that share a provider — and the architectural realization that OAuth scope is a property of a token, not a property of a tool.

The security review said the agent acts "on behalf of" the user. The OAuth token said something else, and the audit log agreed with the token.

A small distinction in language did a lot of architectural work nobody noticed. "On behalf of" is the language a security review reaches for when it wants to capture an arrangement where the agent is a delegate, recognizable as a delegate, and constrained by being a delegate. "Act as" is the runtime behavior when the agent holds a token indistinguishable from the user's own and is therefore the user as far as every downstream system can tell. These two phrases describe completely different threat models. A typical enterprise OAuth integration ships the second one and prices it as the first.

A PII redactor that runs only on inbound traffic is a one-way valve installed at the wrong end of the pipeline. It catches user-submitted names, emails, and account numbers before they reach your logs. It does nothing about the model's own outputs — the place where the same model is now actively assembling text that may contain those same identifiers, drawn from RAG retrievals, tool returns, conversation history, or content the user pasted from another tenant's data. Every team I've watched ship an input-side redactor has a follow-up ticket in the backlog labeled "output-side parity." Most of those tickets never close, because no incident surfaces the gap for six months, and after six months the ticket has accumulated enough re-prioritization to look like a feature request rather than a missing half of a security control.

The failure mode is invariant: input redaction is treated as the canonical control because it is the easier engineering problem and the easier audit story. You wrote a regex set, you ran a labeled benchmark, you proved precision and recall on a fixed corpus, you shipped it behind a feature flag, and the security review accepted it as the PII boundary. The output side has none of that benefit. The model's response is generative, the surface area is unbounded, and the test methodology — "what should it not say in any of infinitely many contexts" — is structurally harder than "what should we strip from a known input." So the team that ships the inlet treats the outlet as future work and the future never arrives until a customer reports another customer's email landing in their transcript.

A Spanish customer complained that her annual renewal had been billed a day early. The support ticket bounced through three queues before it landed in front of an engineer who recognized the smell: a date-formatting regression, European cohort only. He ran git log against the date-formatting module and found nothing. The module had not been touched in eleven days. What had been touched, eleven days earlier, was its package.json — a lodash bump from 4.17.20 to 4.17.22, opened by a security agent, approved by the on-call, merged without comment.

Two lines above the version string, in the same file, was a comment written eighteen months ago: // do not upgrade — breaks the snapshot tests in date-formatting, see FRONT-2418. The security agent had not read it. Or, more precisely: the security agent had read the entire file, but its prompt instructed it to find vulnerable version strings, not to weigh the comments around them. The comment was load-bearing institutional knowledge. The agent treated it as scenery.

This is a coordination failure between two systems that did not know they were colliding. The security agent was doing its job. The original engineer who wrote the comment had done his job. The feature-development agent that respected the pin every time it touched the file was doing its job. Nobody had decided whose job it was to mediate between them.

A support agent gives the right answer to the wrong audience. A customer asks about their account, the model dutifully calls a URL-fetch tool, and a snapshot of that account's context lands on a server the security team has never heard of. No credentials leaked. No API keys exposed. The exfiltration vector was a five-star product review written by a competitor three weeks earlier, retrieved as relevant context because the visible praise actually was relevant to the user's question.

This is the failure mode that breaks the mental model engineers carry from years of web security. The threat model in RAG systems is usually phrased as "we own the corpus" because we own the ingestion pipeline, the embedding model, and the vector database. But owning the code that pulls the content is not the same as owning the content. If your corpus includes any source whose writes are not gated by your authorization, you have handed a prompt-engineering channel to whoever can post.

The IAM role you cut for the agent on day one was supposed to be temporary. The pilot needed momentum, the team needed the agent productive by the demo, and somebody — probably you — added a comment in the PR that said "tighten this after we ship." Ninety days later the pilot has shipped, the agent is in production with paying customers behind it, and the role still has write:* on three buckets the agent has never touched. On-call cannot tell you which of the eighteen scopes are load-bearing and which are vestigial, because the only person who knew is on a different team now, and the runtime telemetry that would prove the difference was never wired up.

This is not a story about a careless team. It is a story about how every team building agents arrives at the same place, because the lifecycle discipline that prevents it has not been invented yet at most companies. Human identity has thirty years of accumulated machinery for this — onboarding workflows, quarterly access reviews, automatic revocation when somebody transfers teams. Agent identity has a Slack message that said "I'll clean this up later." The day-one grant becomes the day-ninety inheritance, and the blast radius scales with every model upgrade, every new tool added to the agent's belt, every customer onboarded against the same role.

A user clicks "Authorize" on your agent's consent screen once. By the time the session ends, that agent has chained through eleven tool calls, negotiated three step-up authorizations, and now holds the union of scopes across every tool it touched. The user remembers granting one thing. Your audit log shows read-write access to half their account. The OAuth standard says everything is working as designed, and that is exactly the problem.

The classical OAuth consent model was built for a world where one app talks to one API. Agents shattered that assumption two years ago and the standard has not caught up in practice, even where the spec has. The result is a category of silent privilege escalation that no one decides to ship — it accretes, one tool registration at a time, while your security review keeps inspecting the front door.

There is an endpoint inside your company that has been safe for ten years. It lives at a path that nobody outside the original team would ever guess. It is not in the public docs. It is not in the OpenAPI spec. It is not in the gateway's allowlist of "documented routes." Its auth layer is a token that any internal service can mint, because the threat model said the only way to reach it was to already know it existed. The endpoint accepts a JSON blob that, on a slow Tuesday, will reissue a refund or rotate an API key or move a row between two billing ledgers. It has worked correctly and unremarkably since 2016.

Last month, a teammate wired a coding agent into the engineering wiki to help with onboarding questions. The agent indexed every Confluence space, every archived design doc, every "do not delete — historical" page. Yesterday, a junior engineer asked it how refunds work. The agent stitched together a forgotten 2018 architecture diagram, a Slack export someone had pasted into a runbook, and a half-written postmortem. It produced, in conversational prose, a complete description of that endpoint, the token type required, and an example payload. The endpoint had not changed. Its threat model had.

A team I know has a textbook staging setup. Read-only replicas of the production database. A mock Stripe account that pretends to charge cards. Synthetic users with fake email addresses on a domain nobody owns. The agent is asked to walk through an "account delinquent" escalation flow in staging, end to end, as part of a release rehearsal. The trace looks clean. The agent does what it is supposed to do.

Three minutes later, a real customer — a paying one, who churned six months ago and was still in a dormant export the developer had used to seed a test fixture — replies to a politely-worded payment-overdue email. The "send_email" tool, registered next to a dozen other tools that all terminate in mocks, was wired to the production Mailgun key. The developer who set it up two sprints earlier had been iterating fast on email templates and the sandbox tier capped them at five emails an hour, which broke the inner loop, so they swapped in the real key "just for the afternoon" and forgot. Nobody re-checked. The agent had no way to know.

The intern arrives on a Monday. By Tuesday afternoon she has wired up her first agent. By Wednesday morning that agent has invoked a production tool through a credential she should not have inherited, and nobody on the security team knows it happened because the audit trail records the call as coming from "the intern's senior mentor's setup script" — which is technically true and operationally useless.

This is not a story about a bad intern or a careless mentor. It is a story about an onboarding pipeline that has decades of refinement behind its assumptions about new humans — read-only first, sandboxed write next, production after a tenure threshold — and zero refinement behind its assumptions about the agents those humans configure on day one. The IAM model for humans is no longer the IAM model for what gets executed against your systems, and most security teams have not noticed yet.