{"id":1013,"date":"2026-04-13T11:10:54","date_gmt":"2026-04-13T11:10:54","guid":{"rendered":"https:\/\/3nayan.in\/blog\/?p=1013"},"modified":"2026-03-31T11:36:23","modified_gmt":"2026-03-31T11:36:23","slug":"ai-roi-measurement-wrong-story","status":"publish","type":"post","link":"https:\/\/3nayan.in\/blog\/2026\/04\/13\/ai-roi-measurement-wrong-story\/","title":{"rendered":"Your AI ROI Numbers Are Telling You the Wrong Story"},"content":{"rendered":"\n

Part of the series: AI at the Wrong Speed<\/em><\/p>\n\n\n\n

\n\n\n\n

What do you think is the usual answer to a board question of whether an AI implementation is working? Almost invariably, it is a number e.g. Cost savings, % Productivity improvement, Tickets deflected, Headcount avoided etc. But, these real and measurable numbers are absolutely the wrong things to look at. <\/em><\/p>\n\n\n\n

The problem with using ROI snapshots to evaluate AI implementation is that they measure the wrong moment, the wrong state of the system. A P&L impact figure tells you where the organisation stood at the point of measurement, making it a lagging indicator. It says nothing about whether the organisation is becoming more capable, more adaptive, or able to extract more value from AI over time. That is an intercept, not a slope. <\/p>\n\n\n\n

And that slope is what determines outcomes in the longer term.<\/p>\n\n\n\n

Why Boards Pull the Plug<\/h2>\n\n\n\n

Let us understanding why AI implementations are reviewed. This will tell us why it leads to AI failures.<\/p>\n\n\n\n

Most AI pilots are evaluated in isolation. Usually, use cases are selected, solutions deployed, measurement periods are defined, and results produced. That result is then compared to the investment. If the ratio is unfavourable, or if the pilot has not produced a visible P&L impact within the expected temporal window, the programme is reviewed, scaled back, or stopped.<\/p>\n\n\n\n

This evaluation model has several fundamental and structural problems. First, the pilot is rarely connected to a full process chain. An AI model that improves one step in a multi-step workflow produces a result that is real but not operationally significant. The overall process does not move faster or better. There isn’t a marked difference in customer data (c-sat, engagement, stickiness, conversion etc.). The cost saving isn’t enough to show up in accounts because it gets disguised by the surrounding process absorbs it.<\/p>\n\n\n\n

Second, the measurement window is almost always too short. AI implementations that are genuinely transformative require an organisational learning curve. This could be data quality which improves over time or employee adoption deepens. Similarly, process redesign, which initially slows things down, eventually produces compounding efficiency. None of this is visible in a quarterly review.<\/p>\n\n\n\n

Third, and the most consequential, the budget has usually been sized for the deployment, not for the full transformation cycle. When costs run ahead of visible returns, the progamme appears to be failing. In most genuine transformations, especially in the middle phases, the costs are front loaded. The CIO, and the boards measure the intercept and see costs accumulating without corresponding output. The “rational” response, is to stop the programme.<\/p>\n\n\n\n

Jim Collins documented a version of this in\u00a0Good to Great<\/em><\/a>: companies celebrated for dramatic performance at a specific point in time frequently did not sustain it. Because of no substance in the trajectory, the momentary euphoria invariably fizzled. What most AI programme reviews are measuring is the intercept of the transformation, a snapshot. What they should be measuring is whether the slope is building.<\/p>\n\n\n\n

The Three Gradients That Actually Matter<\/h2>\n\n\n\n

The below three financial gradients, instead of a vanialla P&L, give a more accurate picture of whether an AI implementation is institutionalising or merely experimenting.<\/p>\n\n\n\n

\"AI<\/figure>\n\n\n\n

Workflow asset turns.<\/strong>\u00a0e.g. task automation rate multiplied by quality score is a better measure than drop is headcount. Is AI handling a growing proportion of defined tasks, and is it doing so with improving accuracy over time? A rising curve here indicates that the organisation is genuinely absorbing AI into its operating model, not just running a parallel pilot.<\/p>\n\n\n\n

Margin-volume divergence.<\/strong> As AI takes over commodity tasks, human effort should shift toward higher-value work. The signal to track is AI-augmented output value per full-time equivalent compared to a pre-implementation baseline. If this ratio is improving, AI is shifting the organisation up the value curve. If it is flat or declining, the implementation is adding cost without repositioning capability.<\/p>\n\n\n\n

Workflow cycle time compression.<\/strong>\u00a0This is a decent proxy for working capital efficiency that most organisations overlook entirely. AI that is genuinely embedded in operational processes reduces throughput time, for instance, faster decision cycles, shorter processing windows, quicker exception handling etc. This compression reduces working capital friction in ways that do not immediately show up on a P&L but are significant at scale. Tracking cycle time across AI-enabled workflows, compared to baseline, reveals whether the implementation is hardening into the organisation’s operating fabric or sitting alongside it.<\/p>\n\n\n\n

The Measurement Shift Required<\/h2>\n\n\n\n

None of these three gradients are complicated to track. What the boards really need to be asking is “is our AI capability compounding?”<\/p>\n\n\n\n

That shift in measurement philosophy is, practically, a shift in governance. This needs:<\/p>\n\n\n\n