# Measuring AI coding assistants

> What Copilot, Claude Code and Codex figures are good for, how to value the time they save, and the pitfalls that make them mislead.
>
> Source: https://roiva.ai/docs/measuring-ai-coding-assistants
> Checked against the product on October 14, 2026

GitHub Copilot, Claude Code and OpenAI's Codex each report how much developers used them, such as suggestions offered and accepted, lines of code, and how many people used the tool on an average day. Not every tool reports every one of these. This page covers what those figures can tell you, how Roiva turns them into a value, and where they mislead.

#### What the figures are good for

- **Valuing the time an assistant saves.** The AI Coding Assistant Rollout template values it as accepted suggestions × minutes saved per acceptance × your fully loaded developer hourly rate. For an assistant that counts no acceptances, such as Codex, it uses lines of code kept × minutes saved per line instead. An initiative gets one of the two, never both, because both measure the same saved hour.
- **Following adoption.** Active users and accepted suggestions, month to month, show whether a rollout is taking hold, slowing, or concentrated in a few people.
- **Comparing a tool with itself.** The same assistant's figures this quarter against last quarter answer most of what a team wants to know.

The value is freed capacity, not cash: developer hours priced at what they cost. It's reported beside cash value and counts in no ROI until someone shows, with a document, that the time became cash, such as a contractor not renewed or a hire not made. See [How Roiva calculates value](https://roiva.ai/docs/how-value-is-calculated).

#### The units aren't the same across tools

Each assistant counts in its own unit, and they don't convert:

- **Copilot** counts acceptance activities. One activity can bundle several completions.
- **Claude Code** counts accepted tool actions, one edit or write each, whether it reports through an Anthropic Console organization or across Claude Enterprise seats.
- **Codex** reports no count of accepted suggestions at all, so Roiva reads the lines of code it added that developers kept.

So Roiva reads coding assistants from one source at a time, the primary, and never adds two tools' counts together. **Integrations → Sources** shows which one the account reads. Where a team runs Copilot and Claude Code side by side and both report, the template values each separately, at its own minutes saved per acceptance, and adds the minutes rather than the counts.

Lines of code differ too. Copilot counts the lines developers accepted, Claude Code the lines it added, and Codex the lines it added that developers kept. Don't add them up or compare one tool's lines with another's.

#### Best practices

- **Calibrate the minutes.** Roiva ships a default for minutes saved per acceptance and per line so a figure appears on day one, but no published study measures either one, so each is a convention rather than a measurement. Time a sample of tasks with the assistant and without it, and replace the default with your own figure. See [Running a time study](https://roiva.ai/docs/running-a-time-study), and the defaults themselves in [Assumption defaults](https://roiva.ai/docs/reference/assumptions).
- **Price Copilot and Claude Code separately.** An accepted Claude Code edit is a smaller unit than a Copilot acceptance, so one minutes figure for both overvalues one tool or undervalues the other.
- **Use a fully loaded rate.** The developer hourly rate should include benefits and overhead, not salary alone, or the saving reads low.
- **Read counts beside active users.** A month with twice the acceptances might mean twice the developers rather than twice the help per developer.
- **Prefer acceptances to lines.** An accepted suggestion is a decision a developer made. A line is output, and says nothing about how hard it was to write.
- **Look for the saving downstream.** If time is really being saved, it tends to show up somewhere a team already watches, such as merged pull requests or cycle time, which GitHub reports too. Those figures have many causes, so treat them as a check, not as the value.

#### Pitfalls

- **Accepted isn't shipped.** An accepted suggestion can be rewritten or deleted an hour later. The count says a developer took the code, not that it survived review.
- **Usage isn't value.** Acceptance counts rise as people try a tool, and stay high when the tool is used for work that wouldn't have taken long anyway. The minutes figure is where that's accounted for, which is why it's worth timing.
- **Seats aren't users.** Licenses bought and people using the tool on an average day are different numbers. Cost follows seats, value follows use.
- **One population, not two.** If some developers use Claude Code through an Anthropic Console organization and others on Claude Enterprise seats, Roiva reads one of the two, not both.

Each of these figures has its own entry in the [Metric registry](https://roiva.ai/docs/reference/metrics), with what to watch out for when you read it.