2 posts tagged with "pii"

A team stands up a fine-tuned redaction model in front of their log pipeline. It strips names, emails, account numbers, and IP addresses before anything lands in long-term storage. The model is small, fast, and easy to deploy alongside the ingestion workers. The privacy review approves it. Six months later a customer support engineer pastes a strange-looking log line into a debugging tool, and the redactor produces an output that contains a real customer's email address — one that does not appear anywhere in the input.

The pipeline did exactly what it was built to do. The redactor was the leak.

A privacy review approves your redaction layer. Names, emails, account numbers, phone numbers — all scrubbed before the prompt reaches the model. Your single-turn classifier still hits 94% accuracy. Six weeks later your multi-step agent starts giving confidently wrong answers to questions like "is the email Sarah used to log in the same as the one on her billing record?" and nobody can reproduce it in dev.

The redaction layer did exactly what infosec asked it to do. It also quietly destroyed the property your agent's reasoning depended on: that two mentions of the same entity in different turns refer to the same thing. The agent isn't hallucinating. It's reading a transcript where Sarah has become three different people and the "same" email address has become two distinct placeholders.