Don't let your AI tell you the value of your AI

To measure what your AI investments are getting you, you need to do more than ask your AI tools — even if those tools are connected to your data warehouse and all your systems.

If you ask an AI assistant what your AI support agent saved last quarter, you'll get a number. But to produce it, the assistant will decide which months count as the baseline, which tickets count as the AI's, and what an hour of an agent's time costs. Those are the calls a CFO will question, and nobody in your business made them.

What does "auditable" mean for an AI ROI number?

An "auditable" assessment of AI ROI means a human-led process producing numbers that other humans can reproduce, and judgments that other humans can evaluate.

Both halves matter. Reproducible arithmetic with undocumented judgments behind it is still unauditable, because the part a CFO will question is the judgment, not the multiplication.

What does a human-led process involve?

Three stages: decisions made before any numbers exist, measurement that is mostly mechanical, and defending the numbers afterward. The judgment sits at the beginning and the end, with the machinery in between.

Stage	Layers	Who does the work
Decisions	Scope, metrics, attribution rules, formulas	A person, before any number exists
Measuring	Baseline, assumptions, observations, cost allocation	Mostly systems, with cost as the exception
Defending	Approvals, limitations	A person in the business who can say no

What has to be decided before any numbers exist?

Everything that involves a choice. All four of these should be written down before you measure anything, because a choice made after you have seen the result is difficult to defend.

Scope: what will be measured? One lighthouse AI initiative, all AI initiatives across engineering and finance? How long will you observe expected change — 3 months, 6 months? Starting with one initiative is usually easier to defend than a portfolio-wide number produced in the same window.

Metrics: what is each initiative supposed to move, and can you actually measure it? Cost per resolved ticket, handling time, win rate. A metric nobody currently reports is one you will be reconstructing later, under time pressure.

Attribution rules: what rules will you use per metric? For an AI support agent, this might be: "We count savings only on tickets the AI agent closed without a human, at the baseline cost per ticket."

The rule should also say whether the savings are cash or capacity. If the work took fewer hours but payroll and contractor spend didn't change, that is capacity freed rather than cash saved, and the board should hear it called that. An avoided hire sits in capacity too, unless there is a decision with a date on it — an approved requisition that was canceled, or a contract that wasn't renewed. "We didn't need to hire" on its own can't be distinguished from a hire that was never really going to happen.

Formulas: the attribution rules made executable. Rules are the decisions; formulas are the versioned expressions that implement them, so last quarter's numbers can be recomputed on last quarter's arithmetic. Like the rules, formulas tell you what data you will need to collect.

What part of the measurement is mechanical?

Most of it. Once the rules and formulas exist, collecting the inputs is largely a systems problem, with one exception.

Baseline: document the baseline figure for every metric, including the reporting period and how it was measured or estimated. Label as an estimate where it is one. Most organizations don't have a baseline sitting ready, so it gets reconstructed from whatever history exists before the initiative went live. That is workable, as long as the reconstruction is written down and the result is labeled for what it is.

Assumptions: date and source every assumption. For example, an average loaded hourly rate based on last quarter, determined by a specific person. An assumption without a date is one nobody can confirm is still true, and rates, prices and mixes all move.

Observations: ensure that observations for your selected metrics are recorded under the same criteria (reporting period, business scope) as the baseline. This is the quiet failure mode covered in Getting to AI ROI in six weeks: a baseline taken from all tickets and an after taken from tier 1 only will show a drop the AI had nothing to do with.

Cost allocation and reconciliation: the risk of double-counting, incorrect allocation, and reconciliation challenges increases with the number and complexity of systems involved. A single vendor bill may cover several initiatives, or a shared platform, or spend that belongs to nobody in particular. AI can help match bill lines to initiatives, but how a shared bill gets split is a decision, and someone should own it.

How do you defend the numbers?

Someone in the business approves the value, and someone records what the number doesn't cover.

Approvals: a human working in the business needs to approve observed value — improvements over the baseline. This needs to be someone who can poke holes. Is that value actually from a negotiated price discount with a vendor? Where are there other confounding variables in the business? Did volume move for reasons that have nothing to do with the AI?

Costs should be deducted from preliminary ROI figures as they come in, and value only added after it has been approved. Costs still need someone to review them — is the allocation right, is anything counted twice? — but they shouldn't wait for that review to count. The asymmetry is deliberate: the number should be harder to inflate than to deflate.

Limitations: a human who understands the business needs to record limitations — spend outside connected systems, estimates that stay estimates, potential confounding variables.

Could someone else reproduce your number?

That is the test the whole process exists for. After this process, other team members should be able to reproduce the ROI numbers using the same assumptions, versioned formulas, and frozen inputs.

If reproducing it requires a conversation with the person who built it, the number isn't auditable yet. It may still be right, but it won't survive that person leaving, and it won't survive a challenge in a board meeting they aren't in.

What does this process not prove?

That the AI caused the result. Attribution is a rule someone chose, not an experiment, and a process like this doesn't rule out everything else that changed in the same period. Spend outside connected systems stays invisible, and estimates stay estimates however carefully they are labeled.

That is worth stating plainly in the board pack rather than leaving it to be discovered.

Six questions to ask about your own number

Question	What a good answer looks like
Could someone else reproduce this from what's written down?	They can, without asking you
Which figures were measured, and which were assumed?	Every assumption has a date and a source
Was the attribution rule written before the result was known?	Yes, and you can show when
Is the baseline documented, including how it was estimated?	Window, method, and labeled as an estimate
Who approved the value, and could they have said no?	Someone in the business, not the project owner
What does this number not prove?	A written list, not a shrug

Where Roiva fits

Roiva is built around this process. Formulas are versioned, inputs are frozen at the point of calculation, value entries move through review and approval before they count in any figure, and every change is recorded in an append-only audit log.

We don't think AI can write your ROI for you, though it can help. Roiva's Assistant can draft a value entry but can't approve one, and our MCP access is read-only, so assistants connecting from outside can read the figures but not change them.

The method is published in full at /methodology, and the mechanics — metrics, baselines, assumptions, allocation rules — are documented at /docs.