Ask an executive whether their AI coding tools are paying off, and you'll usually get one of two answers. "Adoption is up to 85 percent." Or "the developers love it, they say they're much faster."
Neither one answers the question.
McKinsey's latest State of AI survey found that only 39 percent of organizations could point to any effect of AI on their bottom line, and most of those said it was under 5 percent. Most companies have spent real money on AI tools and can't say what they got for it.
That isn't because the tools don't work. It's because most organizations are measuring the wrong things.
What gets measured now
Here's what I usually find when I ask how an organization tracks its AI rollout:
- How many people use the tools. That tells you how much money is being spent and who's spending it. It doesn't tell you what it bought.
- How much code is being produced. Lines of code, commits, pull requests. These go up sharply with AI, which is why they're popular. But a large study of more than 500,000 developers found that commits rose 240 percent while releases rose only 30 percent. More code is not more software.
- How much faster people feel. This is the one most worth worrying about. In a 2025 study by the research group METR, experienced developers took 19 percent longer to finish real tasks with AI tools than without them. Afterward, they believed AI had made them 20 percent faster.
All three measure activity. None of them measures delivery.
What tells you whether it's paying off
The question a CFO actually cares about is simple: is more working software reaching customers, sooner, without more problems? Are we seeing value for our investment? You can answer that with data most organizations already have in their ticketing and release systems.
- Time from idea to customer. Pick a few pieces of work that shipped recently. How long did each take, from the day someone decided to build it to the day customers could use it? If AI is helping, that number should go down.
- Finished work reaching customers each week. Not started, not "in review." Released and in use.
- Work redone after release. How much of what shipped had to be fixed or changed because it wasn't right? If this goes up while output goes up, the speed is costing you.
- Where work waits. For any piece of work, how much of its time was spent being worked on, and how much was spent waiting for review, testing, a decision or a release? In most teams I see, waiting is most of it.
That last one usually explains the other three. AI made the writing faster. If the work then waits just as long for everything else, customers see very little difference.
Start with a baseline
The most common mistake isn't measuring the wrong thing. It's not measuring anything before the rollout. Without a starting point, every result is an opinion.
If you've already rolled AI tools out, you can still build a baseline from history. Your release records and tickets from a year ago show how long work took and how much shipped. Compare those to the last three months. It won't be perfect, but it will be far more honest than a survey.
If you want a quick read on where your teams stand, the Delivery Gap Assessment is ten questions across the five places AI gains usually get lost: where work piles up, how clearly work is described, who makes decisions, who owns the result and how delivery is measured. You'll see your score right away.
Sources
- McKinsey & Company, The State of AI in 2025 (November 2025): 39% of respondents report any enterprise-level EBIT impact from AI; most of those report less than 5%.
- Mert Demirer, Leon Musolff and Liyuan Yang, "Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools," NBER Working Paper 35275 (2026): https://www.nber.org/papers/w35275
- METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" (July 2025): https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/