arrow_backAll articles

ROI of AI in software development: what the studies show and how to measure it in your own company

· Josef Chmel

ROI of AI in software development: what the studies show and how to measure it in your own company

You pay for AI tool licences and your team says it is faster. Yet studies by Google, METR, DORA and Bain show that feeling faster and being faster are two different things. How to measure the ROI of AI honestly.

Most software companies already pay for AI tools for their developers. According to the DORA 2025 report, based on responses from nearly 5,000 technology professionals, 90% of respondents use AI at work and more than 80% believe it has made them more productive. For the head of a software company, though, the question that matters is a different one: what is the real ROI of AI in software development – are we delivering more to customers, faster and at the same quality? Let's look at what independent research says and how to check it in your own company.

What the studies show: from +56% to −19%

Research results diverge more than you would expect:

  • GitHub Copilot, lab task (2023). Sida Peng and colleagues asked developers in a randomized experiment to implement an HTTP server in JavaScript. The group with Copilot finished 55.8% faster than the control group.
  • Google, 96 engineers (2024). A randomized trial on a more complex enterprise task: building a new service in an existing codebase, tests included. The best estimate is a time reduction of about 21%, but with a wide confidence interval – the authors themselves point out that the exact size of the effect is uncertain.
  • METR, 16 experienced open-source developers (early 2025). The developers worked on 246 real issues in their own large repositories; for each issue it was randomly decided whether AI was allowed. With AI, the work took them 19% longer.

Three studies on how AI affects development speed: GitHub Copilot 55.8% faster, Google about 21% faster, METR 19% slower

It makes no sense to read the gap as "AI works / AI doesn't work". A greenfield lab task is ideal for AI; a large, mature codebase with high quality standards is the hardest case. In February 2026, METR also published that newer data points to a speed-up instead (for the original developers an estimated 18% reduction in time), but with an interval so wide that METR itself called it only very weak evidence. The reason is telling: many developers refused to work without AI, so the tasks best suited to AI never made it into the experiment.

Feeling is not measuring

The most important lesson for management is not the slowdown itself. Before the METR study, developers expected AI to speed them up by 24%. Even after the study, when they had in fact been slower, they believed AI had sped them up by 20%.

METR study: developers expected a 24% speed-up, believed afterwards they had been 20% faster, and actually took 19% longer

For the head of a software company this gives a simple rule: a satisfaction survey is not an ROI measurement. The fact that the team likes its AI tools is good news for adoption, but it tells you nothing about how much work you deliver and at what cost. The DORA 2025 report also notes that 30% of respondents have little or no trust in AI-generated code – that, too, is time spent on checking, and it vanishes in surveys of how people feel.

Where the speed-up gets lost on the way to the customer

In its Technology Report 2025, Bain & Company points to simple arithmetic: writing and testing code accounts for only about 25–35% of the time from initial idea to product launch. If AI speeds up only that part, the whole moves little. According to Bain, teams using AI assistants typically see 10–15% productivity boosts – and often don't redirect even that saved time to higher-value work. Companies that pair AI with an end-to-end redesign of the development process report 25–30%.

Bain Technology Report 2025: writing and testing code is 25–35% of the time from idea to launch, AI assistants bring 10–15%, an end-to-end process redesign 25–30%

The second place where the gains get lost is stability. DORA 2024 research estimated that a 25% increase in AI adoption is associated with a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability. In DORA 2025 the relationship between AI and throughput turned positive, but the negative link to stability remains. DORA sums it up this way: AI doesn't fix a team, it amplifies what's already there. Without strong automated testing, mature version control and fast feedback loops, more change leads to more failures in production.

How to measure the ROI of AI in your own company

You don't have to wait for the next study. A few disciplined steps are enough:

  1. Measure the baseline. Before rolling tools out further, record for the past months the time from task assignment to deployment, the number of tasks completed per sprint and the share of defects that came back from production.
  2. Measure finished tasks, not lines of code. ROI lies in delivered value – tasks that passed tests and review and reached the customer. The number of generated lines or prompts tells you nothing.
  3. Count all the costs. Licences and model usage, but also senior time spent on review and fixes. If AI saves an hour of writing and adds an hour of checking, the return is zero.
  4. Decide up front where the saved time goes. Bain recommends this explicitly: more features, lower spending or faster innovation. Without a decision, the saved time simply dissolves.
  5. Watch stability. Track returned defects and failed releases as carefully as speed. A speed-up that comes back as bugs is not a speed-up.
  6. Start with small, well-specified tasks. A task with clear acceptance criteria and tests can be measured and checked. A vague brief can't be solved well by a human or by AI.

How mcptask.online helps

To calculate ROI, you need work and time in one place. In mcptask.online every task has scrum points from which an hour estimate is calculated automatically, and time is logged directly against tasks – including from commits. Timesheets in JSON, CSV or PDF then show what finished tasks really cost, so you can compare before and after you introduce AI.

If you want to go further than an assistant in the editor, mcptask.online offers an autonomous AI developer (the runner). It takes tasks from the same queue as your people, runs on your own machine against your database and tests, and works under its own user identity – so its work is visible separately from the work of humans. It writes code and tests, and in auto-merge mode the pull request is merged only after green tests and CI; otherwise it waits for a human review. The daily budget is read live and the run stops once it is used up, so model costs stay under control. The runner works with Claude Code, Codex CLI and OpenCode, and with the models you choose.

Want to measure the ROI of AI on finished tasks rather than on feelings? Sign up for mcptask.online – the first 30 days are free, no credit card required.

Sources