39 posts tagged with "engineering-leadership"

Most enterprise LLM contracts get negotiated backwards. Legal spends three weeks on the limitation-of-liability section — which will almost certainly never be invoked — while the clause that determines whether your product survives next year gets waved through in the boilerplate: the vendor's right to deprecate the model you built on, with notice defined as "commercially reasonable efforts to inform you."

Ask anyone who had production traffic on a model that got a retirement date. The migration wasn't a legal event; it was an engineering fire drill — re-running eval suites, re-tuning prompts that silently behaved differently, and explaining to customers why the assistant's tone changed overnight. No indemnity clause covers that. But a deprecation-notice clause, negotiated up front, converts the fire drill into a scheduled project.

This is a checklist of the clauses that actually move risk for an AI buyer, roughly ordered by how often they bite — and, just as important, a calibration of which levers vendors will genuinely move on versus the ones where you're burning negotiation capital for nothing.

Somewhere in your company there is a slide with one number on it. "AI quality: 87." Last quarter it said 85, so the slide is green. Meanwhile, your on-call channel is full of screenshots of the assistant confidently inventing refund policies for your largest enterprise customer. Both things are true at once, and the slide is the one lying.

The executive request behind that slide is completely reasonable: give me one score I can track, so I know whether the thing is getting better or worse. It works for revenue. It works for uptime. It does not work for an AI feature, because an AI feature's quality is not a scalar — it is a distribution over inputs, users, and time. Averaging that distribution into a single number doesn't summarize it; it destroys precisely the information a decision-maker needs.

This post is about the gap between those two facts: why the mean of your eval suite hides the regressions that actually hurt you, what to report instead, and how to present a legitimately noisy metric to a board-deck audience without torching your credibility the first week it moves down.

Pull up the architecture diagram for your multi-agent system. Now pull up your org chart. If you squint, they're the same picture. The "research agent" maps to the team that owns search. The "billing agent" has a hard boundary exactly where Finance stops talking to Product. The orchestrator that fans work out to five specialists looks suspiciously like an engineering manager with five direct reports. You didn't decide this on purpose. Conway's Law decided it for you.

Melvin Conway's 1967 observation is that any system you design will mirror the communication structure of the organization that built it. For sixty years this was a story about microservices and monoliths. But agent fleets are the most literal demonstration of the law I've ever seen: the agents are communication structures. An agent boundary is a place where one process hands a message to another and waits. When you draw those boundaries to match your teams instead of your problem, you don't just inherit your org chart's shape — you inherit its dysfunction, and you run it at machine speed.

Your agent just issued a $40,000 refund to the wrong account, re-routed a freight order that triggered expedited shipping fees, or pushed a config change that took down a customer's production environment for six hours. The action is done. It is irreversible, or close enough that reversing it costs real money. Now the only question that matters is the one nobody asked before you shipped the thing: whose budget eats the loss?

Most teams discover the answer the hard way, in a conference room three days later, with the vendor's account manager on speakerphone reading a liability cap back to them. The cap is the annual subscription fee. The loss is forty times that. The conversation is short.

A real regression burned for two days in production. The page had fired. It had fired correctly, at the right threshold, with the right severity. Three weeks earlier the on-call rotation had added a silence rule for that alert family because every page in that family had so far resolved with the same comment: "nothing to do, investigating." The post-mortem could not honestly call the silence a mistake. It was a rational adaptation to a stream of pages the rotation had no playbook for. The regression that mattered shipped against a muted channel because the team's monitoring stack was producing signals it could not act on, and the team's response to that was the only one available: stop listening.

This is not an alerting bug. It is a structural property of how AI features get instrumented when teams reach for the playbook they already know. Latency, error rate, refusal rate, output schema conformance, judge-eval drift — each one is a defensible metric. Each one fires with the same diffuse "model behavior changed" wording. None of them tells the on-call engineer what to do, because no one has written the runbook that maps each signal to an action, because most of the time the signal does not map to an action. The rotation absorbs the noise until the noise is louder than the signal, then it routes around the channel that produces it.

The candidate hit the wall on the system-design question, paused for two seconds, and said: "I'd just prompt it." Your most senior interviewer wrote strong hire — this is exactly how good engineers work in 2026. Your second-most-senior interviewer wrote no hire — handing the problem to a chatbot is not engineering. Same five words. Same forty-minute window. A forty-point gap on the same scorecard.

The candidate didn't fail your loop. Your loop failed to have an opinion. And the worst part of the debrief is not the disagreement — it's the way each interviewer is so confident their read is the correct one that the meeting devolves into a referendum on AI itself rather than on whether this human can ship.

The customer pasted the agent's answer into a support chat and asked the human rep to confirm it. The rep, looking at the same product, said the opposite. The customer did not lose trust in the agent that day. They lost trust in the company, because two parts of it told them two different things in the same hour.

Nothing was broken. The AI team had shipped a prompt change on Tuesday behind a feature flag, ramped it to 100% by Thursday, and moved on. The customer success team's enablement cycle is monthly — that is how every other product feature has always landed, and nobody re-negotiated the contract for AI. The macro in the CS rep's queue and the FAQ doc on the public site still described the previous behavior. The agent was correct. The rep was correct against the documentation they had. The company was incoherent.

The post-mortem says "we found security too late." The actual finding is that security found you on time. Your process found security too late.

This is the AI feature that cleared the budget gate in January because the CTO and the CFO agreed the company needed an AI moment. It cleared a light legal review in March because it was a prototype. Engineering built against the agreed spec through Q2. In late July, the launch-readiness security review opened, and on day one the threat model came back with blockers on the auth scopes, the data-exfiltration paths, the model provider's residency story, and the prompt-injection surface. The team's quarter is now spent rebuilding to address findings that should have shaped the original spec. Two quarters of slip, an executive memo about "process improvements," and a quiet decision next planning cycle to "deprioritize AI deep-integrations."

The launch did not fail because security was slow. It failed because security entered after the shape of the feature had already been frozen.

You sketched a six-quarter AI roadmap. The model swap, the new data source, the multilingual launch, and the prompt that now offers advice each got a single row on the Gantt chart, sized by engineering effort. Then the first launch slipped four weeks, and the post-mortem said the same thing three times in three different sections: "waiting on legal." The roadmap had assumed engineering capacity was the binding constraint. The actual binding constraint was a queue of legal reviews, each running its own three-to-six-week SLA, none of them aware of each other, and all of them landing on the same two product counsels.

The mistake was not in any of the individual reviews. Each one was warranted. The mistake was treating four parallel features as four parallel timelines while their legal dependencies serialized through the same upstream resource. By the second slip the org learns the shape of the problem. By the fourth it learns to plan against it. The teams that ship AI features on a predictable cadence have stopped treating legal throughput as an external surprise and started treating it as a planning input on the same footing as headcount and infra capacity.

The team meets at 10am. The first engineer reports what their agents finished overnight. Except the eval suite that kicked off at 7am hasn't returned, the PR the agent opened at 3am is waiting on a review from another agent whose queue depth is unknown, and the long-running refactor agent is on hour eleven of an estimated four-hour run with no signal that it's stuck and no signal that it's healthy. Yesterday's status is not "done" and not "in progress." Yesterday's status is unknowable from inside the room.

The standup was a synchronous ritual built for synchronous human work. Each person did a thing, finished it, slept on it, and reported it the next morning. The unit of work was a workday. The unit of reporting was a person. The cadence matched the substrate. None of that holds anymore. The unit of work is now an agent run that started before you went to bed and may finish during the meeting or three hours after. The unit of reporting is a fleet, not a person. And the cadence — a 9- to 15-minute round-robin at 10am sharp — is a frequency the substrate doesn't produce events on.

Your strongest AI engineer spent the cycle curating an eval set, calibrating a judge prompt, and killing two features that turned out to be task-shape mismatched. None of that fits a single line on the review template. So the calibration meeting either inflates the artifacts the engineer cares least about — PR count, design docs, on-call shifts — or invents prose to justify a high rating the framework cannot defend. Either way, the rubric and the reality are pulling in different directions, and the engineer can tell.

The template was written for deterministic software. It rewards what you can count: lines of code shipped, services owned, incidents resolved, hours spent on-call. The AI roadmap is moved by a different shape of work: curating a representative eval slice, defending a behavioral envelope under model drift, refusing to ship a feature whose task shape doesn't fit the model, and patiently shrinking the gap between a judge prompt and human intent. Almost none of that produces the artifacts the rubric was built to count.

Somewhere in your company, four people each believe a fifth person owns the inference bill. Engineering treats it as a cloud line item. The AI team treats it as the price of building. Finance treats it as a variable margin input that someone in engineering must already be managing. Product treats it as overhead that engineering absorbs. The bill keeps growing, and the only thing everyone agrees on is that it isn't theirs.

This is not a budgeting problem. It is an ownership vacuum, and it surfaces the first time the line item gets large enough for a CFO to ask about it on a board call. By then, the answers people improvise — "we'll optimize," "we'll cache more," "we'll switch models" — describe interventions without naming an owner. The conversation that should have happened a year earlier was not about how to lower the bill. It was about whose P&L the bill belonged to in the first place.

The shift is structural. Inference moved from 15% of enterprise AI spend in 2024 to roughly 85% in 2026, and the average enterprise AI budget grew from $1.2M to around $7M over the same window. A line item that was once rounding error is now the kind of number a board notices, and the org chart written before that shift has no row for it.