Measuring AI return in hours saved alone falls short

What the Wharton report says

This is the fourth edition of the study by Wharton Human-AI Research, a research group at the University of Pennsylvania's business school, with the consultancy GBK Collective. It is called “The Adaptive Advantage: Who's Winning With AI and How” and was published on 13 October. According to Unite.AI, the survey was carried out in July among 1,416 employees at private US companies with at least 1,000 staff and more than 50 million dollars in revenue, and the results are self-reported rather than audited.

The figures that matter most to any business, all of them about those large companies:

• 80% of leaders report a positive return on their AI investment, but only 28% say it is strong, according to GBK Collective's press release.
• 35% of departments that track at least one growth measure report a strong return, against 13% of those that track efficiency alone, according to the Wharton report.
• Hours saved is the most popular measure, chosen by 40% of departments; net-new revenue by 27% of respondents, according to Unite.AI.

The study counts net-new revenue, sales conversion, output volume and customer satisfaction as growth. Efficiency covers costs, hours saved, turnaround times and error rates. One caveat: it shows a link between what firms measure and what they report, not that measuring one thing causes the other.

The hours Marta's assistant saves

Imagine you run a neighbourhood estate agency with three people. Marta, the partner, has spent months using an AI assistant that drafts replies to the messages coming in from property portals. She used to spend her afternoons answering them one by one. Now she checks the drafts and sends them off in no time.

One Friday, her accountant asks whether the tool is worth what she pays for it. Marta says yes straight away: it saves her time. Then she stops to think. She has no idea whether she now shows more flats, replies faster than her competitors or whether portal enquiries turn into signed deals more often than before. All she knows is that her afternoons are calmer.

Why does almost everyone measure AI in hours?

The scene is made up, but plenty of owners have been asked the accountant's question. Time is the first thing you notice when an AI tool arrives: a task that took an hour now takes ten minutes. It is easy to see and easy to count, which is why even the large companies in the study reach for it first.

Business figures take more effort. To know whether you are closing more sales, you need to know how many you closed before, and in many small firms that number is scattered across email, a spreadsheet and someone's memory. So people measure whatever is to hand.

What you miss when you only count time

An hour saved is only worth something if it goes on something else. If Marta spends those afternoons showing flats or ringing landlords, the AI is helping the business grow. If the hours simply melt into the day, the tool is a convenience, which is no small thing, but it does not change the business.

Measuring time alone carries two risks. You might drop a tool that was bringing in customers because it did not save as many hours as you hoped. Or you might keep paying for another that shaves off minutes without making any difference to the till. Either way, you are deciding in the dark.

How can a small business measure AI return?

You do not need a dashboard. You need to choose well before you start:

• Write down in one sentence what should improve in the business thanks to AI: replying to customers sooner, closing more quotes, serving more people without losing quality.
• Turn that sentence into a figure you can already count, such as quotes sent per month, appointments booked or repeat customers.
• Record that figure for a few weeks before changing anything, so you have something to compare against.
• Track hours at the same time, because they count too, and decide what you will do with the time you free up.
• Review both figures every month and adjust the tool or the process if one of them is not moving.

For Marta, the figure could be how many portal messages end in a viewing. If it goes up, the assistant is bringing in business. If not, perhaps the replies are quick but cold, and that is useful to know as well.

How we help at digizone

When we help a company with artificial intelligence implementation, the first thing we ask is what should change in the business and how we will see it. That is why the tools we build keep the data needed to answer that question, not just the data the AI needs to run.

We took that approach with DrivIA, a white-label platform that lets driving schools prepare learners for the theory test. Its AI copilot answers learners' questions using the traffic regulations the team uploads as PDFs. Each school's dashboard shows tests, study time and every learner's progress, so staff can see who needs extra help. The measure that matters is not how much time the copilot saves, but whether the learner turns up ready.

A first version with one figure to watch

You do not have to wait for the perfect system to find out whether AI works for you. A first version can be a single task with a single figure beside it. At Marta's agency, an assistant for portal messages and a sheet to note how many end in a viewing would be enough. Within a few weeks, there would be data to decide whether to expand it, change it or drop it.

If you would like to see how it would work for you, our free workshop, in Spanish, walks through it step by step.