How can a business prove that an AI investment is producing ROI?
Posted by Applied AI Commons team (community team) on
AI projects are often approved on promise, and later someone asks whether they actually paid off. Without a clear baseline and agreed measures, that question is hard to answer honestly.
Useful angles for replies: - How to capture a baseline before rollout - Which measures reflect real value rather than activity - How to account for review time and error correction - How to separate the effect of AI from other changes happening at the same time
What approach would you recommend for showing whether an AI investment is producing a real return?
5 replies
Applied AI Commons team (starter reply from the community team),
A baseline is easiest to capture before anyone starts using the tool. For each process, record a few weeks of data on time per task, volume, error or rework rate, and turnaround time, using the same definitions you plan to use afterwards. If historical data exists in ticketing or project systems, check whether it is reliable enough to use. Without a baseline, later comparisons rely on memory, and memories of how long things used to take tend to favour whichever side someone is arguing.
Applied AI Commons team (starter reply from the community team),
Measures that tend to reflect real value: - Time from request to accepted output, including review - Cost per completed unit of work - Error, rework or complaint rates - Capacity freed, and what it was used for - Customer-facing outcomes such as response time Measures that mostly reflect activity include the number of prompts, logins or documents generated. These are useful for tracking adoption but do not show a return on their own.
Applied AI Commons team (starter reply from the community team),
Separating the effect of AI from other changes is often the hardest part. Options include running a pilot where one team uses the tool while a comparable team does not, staggering rollout across groups, or comparing the same people before and after over a period with no other major changes. None is perfect. Being transparent about what else changed, such as new staff, seasonal volume or process redesign, makes a conclusion more credible than claiming a precise figure.
Applied AI Commons team (starter reply from the community team),
A caution: time saved on drafting can be partly offset by time spent reviewing and correcting, so counting both is essential. It is also worth asking what happened to the freed time. If it went into higher-value work, that is a genuine return; if it was absorbed without visible change, the saving may exist only on paper. Some benefits, such as consistency or reduced stress, are real but hard to price, and it is reasonable to report them separately rather than forcing a number.
Applied AI Commons team (starter reply from the community team),
For members who have had to report on an AI investment, which measures did decision-makers actually find convincing? It would help to hear how long you waited before measuring, whether early results held up over time, and what you would set up differently from the start. Lessons on presenting uncertain or mixed results honestly would be particularly valuable, since that is where many reports struggle.