Skip to main content

The Model Is Already Talking to Your Customers

· 10 min read
Tian Pan
Software Engineer

Somewhere, right now, an AI assistant is explaining your product to a prospect. It is quoting a price you changed eighteen months ago, recommending an integration you sunset last quarter, and suggesting an API endpoint that returns 410 Gone. You will never see this conversation. No analytics event fires. No session recording exists. The prospect either believes the wrong answer and files a confused support ticket, or believes the wrong answer and quietly buys from the competitor the model mentioned in the same breath.

This is not a hypothetical future problem. AI referrals already account for meaningful traffic — up to 5–8% for some technology and e-commerce sites — and the answers behind those referrals are generated from whatever the model absorbed about you, whenever it absorbed it. Your marketing team spent a decade learning to monitor brand search, review sites, and social mentions. Almost nobody is monitoring the surface that is growing fastest: what the models say when someone asks about you.

The uncomfortable framing is this: the model is a sales rep and a support agent you never hired, never trained, and cannot fire. It works every time zone, answers instantly, and speaks with total confidence. The only lever you have is what it reads.

A Distribution Channel Nobody Provisioned

The scale snuck up on everyone. ChatGPT's web traffic grew 84% between September 2024 and March 2026, Gemini grew roughly ninefold, and the referral mix fragmented fast — ChatGPT went from 89% of measurable B2B AI referrals to about 63%, with Claude, Gemini, and Perplexity splitting the rest. In May 2026, ChatGPT started surfacing clickable brand links directly inside responses, and for many brands, homepage referrals from AI became the majority of their AI traffic almost overnight.

The volume numbers still look small next to Google — AI referrals sit around 0.1% to 2.8% of total traffic for most sites. But two properties make them disproportionately important. First, the traffic converts unusually well, because a user who arrives from an AI answer has already been pitched: the model summarized what you do, compared you to alternatives, and made a recommendation before the click. Second, and more important, the referral is the visible tail of a much larger invisible body. For every user who clicks through, many more simply accept the model's summary and never visit at all. The answer was the interaction.

That second property is the one that should worry you. When the model describes your pricing wrong, there is no bounce rate to notice, no session to replay. The error is distributed across millions of private conversations, and the only symptoms that surface are downstream and hard to attribute: support tickets referencing features you do not have, prospects negotiating against prices you never published, developers filing bugs against endpoints you deprecated years ago.

The Wrong Answers Are Specific, and Some Are Dangerous

It is tempting to file this under generic "hallucination" hand-wringing. The reality is more concrete. The failure modes cluster into patterns, and some of them have security consequences.

The most common category is staleness presented as fact. Models are trained on snapshots. Your current pricing page, your renamed product tiers, your new rate limits — none of it exists in the base weights until a retraining pass or a retrieval step picks it up. The model does not say "as of my last update"; it states the 2024 price as the price.

The second category is plausible fabrication. Research presented at USENIX Security 2025 tested sixteen models across 576,000 code samples and found that hallucinated package names follow predictable patterns: roughly 38% are conflations that mash two real things together, 13% are typo variants, and half are pure fabrications. The same generative habits that invent express-mongoose will invent an endpoint on your API that looks exactly like your naming convention. Developers trust it because it is stylistically correct.

The third category is where it turns hostile. Attackers noticed that models hallucinate consistently, and consistency is exploitable. Slopsquatting — registering the package names models tend to invent, then loading them with credential stealers — is an active attack pattern. Security researchers have documented the same technique with domains: models generate fictitious-but-plausible web portals and API hostnames for real brands, and attackers register them. Your brand's hallucinated surface area is now part of your attack surface, and you did not draw that perimeter.

There is also a legal floor forming under all of this. When Air Canada's own chatbot invented a bereavement refund policy, a British Columbia tribunal rejected the airline's argument that the bot was somehow a separate entity and held the company liable for what it said. That precedent covers your first-party bots. Third-party assistants are murkier — you are not liable for what ChatGPT says about you — but the commercial damage does not care about liability. A lost deal is lost either way, and with a third-party model you do not even get the option to fix the bot. You can only fix what it reads.

Treat Model Answers Like a Monitored Surface

The operational move is to treat "what do the models say about us" exactly the way you treat brand search rankings or status-page uptime: as a surface with owners, baselines, and alerts.

The methodological trap is treating a single answer as the truth. Model answers are non-deterministic — the same prompt yields different responses across sessions, phrasings, and days, and studies of AI citations find that 40–60% of the sources models cite change month to month. One screenshot of ChatGPT getting your pricing wrong is an anecdote. What you want is a distribution.

Loading…
References:Let's stay in touch and Follow me for more thoughts and updates