LISTEN

Listen to this perspective

Written by Oleg Cohen. Companion narration uses the Brian voice.

This week Martin Fowler, one of the most respected voices in software, made an admission that should comfort a lot of business owners. Many of his colleagues, he wrote, live at the “stunning” end of AI. He mostly uses free-tier chatbots, and he finds their unreliability often cancels out their usefulness.

He was responding to Andrej Karpathy, who recently described the world’s experience of AI as a sharp funnel. Most people barely touch these systems. Another large group uses a free chatbot now and then, and to them, in Karpathy’s words, “it looks like a derpy chatbot.” A small slice of professionals watch agents complete work that used to take weeks. And a few thousand people inside the labs work with systems the rest of us haven’t seen. “A question with a paragraph answer,” he wrote, “barely stresses the system.”

So when an owner says AI isn’t much use and an engineer says it changes everything, both can be right. They’ve used different systems on different problems. A leadership team can’t settle what AI means for the company by polling the room, because the people in it have experienced different AIs.

That gap explains a lot of confused conversations. It also hides a second gap, and that one matters more for an operating company.

Two different capabilities

What a model can do and what a business can dependably do are separate things. A stronger model removes a reasoning bottleneck. It leaves the rest of the work where it was: the information the task needs, the authority to act, the coordination between people, the physical steps, and someone checking that it all happened.

Fowler’s notes from a gathering of people at the leading edge of AI put it memorably. One participant said large companies “have acquired a Swiss-army knife, but are only using the toothpick.” The reasons given were familiar: workflows built over years, security concerns, and data that isn’t ready. None of those are model problems. A company can have frontier models and still use the toothpick.

I’ve argued throughout this series that the unit that matters is a business capability, not a tool. Karpathy’s funnel adds a useful warning. The spectacular results at the top come from people with the best models, rich context, strong tools, long execution horizons and fast feedback on whether the work is right. Most operating companies have few of those conditions for most of their work. That isn’t a reason to wait. It’s a description of what has to be built.

“Done” is a claim

Annie Vella, writing about how software engineering is changing, frames it as a move from direct control to supervisory control. Before AI, she writes, “we closed the loop ourselves.” Now agents write, test and fix, and people supervise them. Then she adds the sentence every operator should keep in view: “what an AI agent says it did and what it actually did may not be the same thing.”

In a business, that sentence has teeth. An agent reports that a supplier invoice was corrected. Was the correction posted? Did the goods actually arrive? Was the discrepancy explanation accepted by someone with the authority to accept it? Did the account end up profitable? Each of those is a different claim, and each needs a different kind of check. One is arithmetic. One requires reading the state of another system. One depends on a physical event. One is expert judgment. The last can only be known months later. Verifying one of them doesn’t verify the others.

Karpathy noted that at the frontier, “human review and comprehension are starting to fall behind.” Inside companies, the same thing happens at a smaller scale whenever output grows faster than anyone’s ability to check it. A company that scales production without scaling verification hasn’t gained capacity. It has built a queue and hidden risk inside it.

Count the human work after the agent finishes

Nate Jones recently gave this problem a useful name, productivity theater, and a useful rule: count the human work after the agent finishes.

Generating a proposal, an analysis or a plan is now nearly free. What follows isn’t. Someone has to read it, check it, decide, coordinate, act and often maintain what was created, and that someone is frequently not the person who generated it. Cheap production also suggests more work. Each plausible output proposes another plausible task. Each accepted task creates obligations for someone, and those obligations accumulate across roles and over time. Visible activity rises while decisions and completion slow down. In Busy With AI I wrote that activity is easy to count and capability is not. This is the mechanism behind that.

Two of Jones’s disciplines deserve a place in any company’s operating habits. First, run more experiments than you commit to. Exploration is valuable even when its result is thrown away, but an experiment doesn’t authorize the continuing work it suggests. A finding should have to earn its way into a commitment. Second, decide when a recurring assignment ends before you make it recurring. A weekly agent-produced report can run accurately for months against a purpose that ended in the spring. Nobody stops it, because stopping it was never anyone’s job.

Supervision is a job, not a button

Vella’s conclusion is that “our next step must be to start redesigning the job and the system, in tandem.” That applies well beyond software.

Putting an approve button in front of an agent doesn’t create oversight. The person approving needs to understand what changed, what evidence supports it, what’s uncertain, and what happens if they say no. They need time to respond before the consequence lands. If the action takes effect faster than anyone could react, the veto is decorative. The control has to come earlier, through narrower authority, limits on what the agent may do, or a way to reverse the result. And someone has to be able to take over when the agent can’t finish.

That is job design. It decides who checks, with what information, before which consequences, and who owns the exceptions. In The Missing Owner I argued that owning the outcome, designing the capability and delivering the technology are three different jobs. Supervising the work once it runs is a fourth, and most AI programs never assign it.

Measure the finished outcome

All of this changes what a company should measure. Licenses, prompts, documents generated and hours saved describe activity. The useful unit is the completed, verified outcome: the invoice actually resolved, the quote whose margin held, the exception closed by the right person. Its cost includes review, failed attempts, exceptions, recovery and maintenance, not just model fees.

It also helps to keep three things apart. Released capacity is time freed up. Redeployed capacity is that time actually put to another use. Realized value is a change that shows up in revenue, margin or cash. They often describe the same improvement at different stages, so they can’t be added together. As I wrote in EBITDA Up, Value Down, each saved hour can be counted once.

None of this argues for doing less with AI. It argues for counting the whole job. More human work can be exactly right when it’s profitable demand, better service or necessary learning. The goal is worthwhile work within the capacity the company actually has.

Questions for leadership teams

  • Which AI has each member of your leadership team actually experienced, and whose experience is shaping your decisions?
  • For your most important AI-assisted work, how do you know it’s actually done?
  • Who reviews what the agents produce, and has anyone counted that work?
  • Which recurring AI assignments have outlived their purpose, and whose job is it to notice?

Where I sit: I advise through Kainora and am building technology aimed at this problem: verified execution, clear authority and outcomes the business can measure. I’d rather be held to these questions than exempted from them.

Producing is the cheap part now. The companies that pull ahead will be the ones that get better at finishing.

Sources and further reading

Share your perspectiveConnect with Oleg on LinkedIn (opens in a new tab)