61 posts tagged with "ai"

A clean PDF, a capable LLM, and thirty lines of code. The demo works. You extract the invoice total, the contract dates, the patient diagnosis. Stakeholders are impressed. Then you push to production, and within a week the pipeline is silently returning wrong data on 15% of documents — and nobody knows.

This is the document AI trap. The failure mode isn't a crash or an exception; it's a pipeline that reports success while producing garbage. Building production document extraction is a fundamentally different problem from building a demo, and most teams don't realize this until they've already shipped.

You shipped the AI feature. You put it in the product. You wrote the help doc. And still, six months later, your most sophisticated enterprise users are copy-pasting text into ChatGPT to do the same thing your feature already does natively. This is not a training problem. It is a discoverability problem, and it is one of the most consistent sources of wasted AI investment in enterprise software today.

The pattern is well-documented: 49% of workers report they never use AI in their role, and 74% of companies struggle to scale value from AI deployments. But the interesting failure mode is not the late-adopters who explicitly resist. It is the engaged users who open your product every day, never knowing that the AI capability they would have paid for is sitting one click away from where their cursor already is.

Most enterprise security teams have a reasonably well-developed model for insider threats: a disgruntled employee downloads files to a USB drive, emails a spreadsheet to a personal account, or walks out with credentials. The detection playbook is known — DLP rules, egress monitoring, UEBA baselines. What those playbooks don't account for is the scenario where you handed every one of your employees a tool that can plan, execute, and cover multi-stage operations at machine speed. That's what deploying AI coding assistants and RAG-based document agents actually does.

The problem isn't that these tools are insecure in isolation. It's that they dramatically amplify what a compromised or malicious insider can accomplish in a single session. The average cost of an insider incident has reached $17.4 million per organization annually, and 83% of organizations experienced at least one insider attack in the past year. AI tools don't introduce a new threat category — they multiply the capability of every threat category that already exists.

Slack discovered that teams exchanging 2,000 messages converted to paid at a 93% rate. The insight sounds obvious in retrospect — engaged teams stay — but what's less obvious is the engineering consequence: Slack built their entire onboarding flow around getting teams to that message count, not around feature tours or capability explanations. They taught users about Slack by using Slack.

AI features have the same problem, but harder. There's no equivalent of "send your first message" because the capability surface is invisible. A user staring at a blank prompt box has no intuition about what's possible. This is the magic moment problem: your product has a transformative capability, but users can't imagine it until they've seen it, and they won't see it unless you engineer the path.

The data makes this urgent. In 2024, 17% of companies abandoned most of their AI initiatives. In 2025, that number jumped to 42% — a 147% increase in a single year. The technology improved; the onboarding didn't.

Cursor hit $1 billion in revenue in 2025 and lost$150 million doing it. Every dollar customers paid went straight to LLM API providers, with nothing left for engineering, support, or infrastructure overhead. This wasn't a scaling problem—it was a unit economics problem that was invisible until it was catastrophic.

Most engineering teams building AI features make the same mistake: they treat inference cost as a minor line item, ship a flat-rate subscription, and assume the economics will work out later. They don't. Variable inference costs don't behave like any other COGS in software, and the pricing architectures that work for traditional SaaS will bleed you dry the moment your heaviest users find your most expensive feature.

There is a company charging £50 per month per user. Their AI feature consumes £30 in API fees. That leaves £20 to cover hosting, support, and profit — before accounting for a single refund or churned seat. They built a product users love, grew to thousands of subscribers, and unknowingly constructed a business where more customers means more losses.

This is not a cautionary tale about a bad idea. It is a cautionary tale about a pricing architecture imported from a world where the marginal cost of serving the next user was effectively zero. That world no longer fully applies when your product calls a language model.

Traditional SaaS gross margins run 70–90%. AI-forward companies are reporting 50–60% — and the gap is mostly explained by one line item: inference. When tokens are 20–40% of your cost of goods sold, the standard SaaS playbook inverts.

You'll find out through a screenshot. A customer will post it, a journalist will quote it, or someone on your team will Slack you a link at 11pm. Your AI system said something confidently wrong — wrong enough that it's funny, or wrong enough that it could hurt someone — and now it's public.

Most engineering teams spend months hardening their AI pipelines against this moment, then discover they never planned for what happens after it arrives. They know how to iterate on evals and tune prompts. They don't know who should post the response tweet, what that response should say, or how to tell the difference between a one-off unlucky sample and a latent failure mode that's been running in production for weeks.

This is the playbook for that moment.

An agent with 95% per-step reliability sounds impressive. At 10 steps, you have a 60% chance of success. At 20 steps, it's down to 36%. At 50 steps, you're looking at a coin flip—and that's with a generous 95% estimate. Field data suggests real-world agents fail closer to 20% per action, which means a 100-step task succeeds roughly 0.00002% of the time. This isn't a model quality problem or a prompt engineering problem. It's a compounding math problem, and most teams building agents haven't internalized it yet.

This is the delegation cliff: the point at which adding one more step to an agent's task doesn't linearly increase the chance of failure—it multiplies it.

A leaked internal Google memo put it plainly: "We aren't positioned to win this arms race and neither is OpenAI." The author's argument was that fine-tuning a model with LoRA costs roughly $100, that open-source communities could replicate closed-model capabilities within months, and that "we have no moat." This was a Google researcher writing about Google. If that's true inside the world's best-resourced AI lab, what does it mean for your product team betting on a data advantage?

The honest answer is that most AI features are not moats. They are rented capabilities with a UI. But some genuinely compound — and the difference is not about how much data you have. It's about the specific mechanical conditions under which data actually creates defensibility.

Your model achieved 91% accuracy on the held-out test set. Latency is under 200ms at p95. You've cut the error rate by 40% compared to the previous rule-based system. By every technical measure, the project is a success. Six months later, leadership cancels it.

This is not a hypothetical. Eighty percent of AI projects fail to deliver intended business value, and the majority of those failures are not caused by model performance. They are caused by the gap between what engineers measure and what decision-makers understand. The technical team speaks a language that executives cannot evaluate — and in the absence of comprehensible signal, leadership defaults to skepticism.

The metrics translation problem is not a communication soft skill. It is an engineering discipline that most teams treat as optional until the funding review.

Your system averages 400ms end-to-end. Your p95 is 4.2 seconds. Your p99 is 11 seconds. You committed to a "sub-second" experience in the product spec. Every metric in your dashboard looks fine until someone asks what happened to 5% of users — and suddenly the average you've been celebrating is the thing burying you.

This is the latency budget problem for AI features, and it's categorically different from what you've solved before. When your core component is a database query or a microservice call, p95 latency is roughly predictable and amenable to standard SRE techniques. When your core component is an LLM, the distribution of response times is heavy-tailed, input-dependent, and partially driven by conditions you don't control. You need a different mental model before you can set an honest SLO — let alone hit it.

The teams that stand to gain the most from AI are often the last ones deploying it. A healthcare organization that could use AI to catch medication errors in real time sits at 39% AI adoption, while a software company running AI-powered code review ships at 92%. The ROI differential is not even close — yet the adoption rates are inverted. This is the AI adoption paradox, and it's not an accident.

The instinct is to explain this gap as risk aversion, regulatory fear, or bureaucratic inertia. Those factors exist. But the deeper cause is structural: the accuracy threshold required to unlock value in high-stakes domains is fundamentally higher than what justifies autonomous deployment, and most teams haven't built the architecture to bridge that gap.