Within Code Review
Why Can AI Make Expert Developers Slower?
Experienced maintainers can spend more time validating AI suggestions than making the change themselves.
On this page
- What the METR study measured
- Where review time accumulates
- Why project knowledge changes the equation
Page outline Jump by section
Introduction
AI coding assistants can accelerate some programming tasks, but evidence suggests they may slow down experienced developers when those developers are working inside repositories they already know intimately. The reason is not that the generated code is always poor. Rather, experts often possess deep project knowledge that the AI lacks, forcing them to spend time verifying, correcting and integrating suggestions that are only partially aligned with the repository’s architecture, conventions and hidden assumptions. A major 2025 study by the research organisation METR found that experienced open-source maintainers working on familiar projects completed tasks about 19% more slowly when AI assistance was available, despite expecting substantial speed gains and continuing to feel more productive while using the tools. [Metr]metr.org2025 07 10 early 2025 ai experienced os dev studyMeasuring the Impact of Early-2025 AI on Experienced…10 Jul 2025 — We conduct a randomized controlled trial (RCT) to understand ho…
This finding is especially important when discussing AI coding assistants and the review burden. For expert maintainers, the bottleneck is often no longer code generation itself. It is the cost of deciding whether generated code truly belongs in a complex, long-lived codebase.
What the METR Study Measured
The strongest evidence for expert slowdowns comes from METR’s randomised controlled trial of experienced open-source developers. Researchers recruited 16 developers who worked on mature repositories where they had, on average, around five years of prior experience. Across 246 real development tasks, participants were randomly assigned periods when AI tools were permitted and periods when they were not. The developers primarily used contemporary tools such as Cursor and Claude-based assistants. [arXiv]arxiv.orgMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity…
The expectation before the experiment was clear. Developers predicted that AI would reduce completion time by roughly 24%. Even after completing the tasks, they still believed AI had helped them work faster. Yet measured outcomes showed the opposite: tasks took approximately 19% longer when AI assistance was available. [arXiv+2Reuters]arxiv.orgMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity…
What makes the result noteworthy is the setting. The developers were not novices learning unfamiliar systems. They were maintainers working in repositories they already understood. That distinction helps explain why results differed from earlier studies that reported substantial productivity gains on narrower programming exercises. [arXiv+2Reuters]arxiv.orgMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity…
Where Review Time Accumulates
The slowdown was not caused by AI generating completely unusable code. Instead, time accumulated across several review-related activities that experts could not safely skip. Researchers observed that developers spent less time directly typing code and searching for information, but more time prompting, waiting for responses, reviewing output and cleaning up generated changes. Roughly 9% of total task time was spent reviewing or correcting AI-generated code. [TechRadar+2IT Pro]techradar.comTech Radar Using AI might actually slow down experienced devsConducted on 16 seasoned developers working on 246 tasks across familiar open-source projects, the study revealed that while developers i…
Several forms of review overhead repeatedly appeared:
- Checking hidden assumptions. Generated code might appear correct while violating project-specific architectural decisions.
- Validating correctness beyond compilation. Experts often need to verify behaviour across multiple subsystems, not merely confirm that code runs.
- Removing subtle regressions. AI can introduce duplication, unnecessary abstractions or maintenance problems that are difficult to detect automatically.
- Rewriting generated solutions. A suggestion may solve the immediate problem while conflicting with long-term project direction.
- Confirming repository conventions. Naming patterns, dependency policies and testing expectations frequently exist outside the AI’s understanding. [Metr+2Reuters]metr.org2025 07 10 early 2025 ai experienced os dev studyMeasuring the Impact of Early-2025 AI on Experienced…10 Jul 2025 — We conduct a randomized controlled trial (RCT) to understand ho…
An important detail from the study is that fewer than 44% of AI suggestions were accepted as generated. Many outputs required significant modification before they could be merged into the project. [TechRadar+2IT Pro]techradar.comTech Radar Using AI might actually slow down experienced devsConducted on 16 seasoned developers working on 246 tasks across familiar open-source projects, the study revealed that while developers i…
For an expert who already knows how to implement a change, reviewing and repairing an almost-correct solution can take longer than writing the solution directly.
Why Project Knowledge Changes the Equation
The key difference between beginners and experienced maintainers is the value of tacit knowledge.
Large software repositories contain extensive information that is not fully documented: historical design decisions, known edge cases, compatibility concerns, performance trade-offs and social expectations among contributors. Experienced maintainers carry much of this knowledge in their heads. AI systems generally do not. [IT Pro]itpro.comIT Pro Think AI coding tools are speeding up work?Think again - they're actually slowing developers downA recent study by Model Evaluation & Threat Research (METR) challenges common assum…
As a result, experts often face a paradox. The more they know about the repository, the easier it becomes for them to spot shortcomings in generated code. Every detected issue creates additional review work.
Consider a maintainer who knows that a particular module exists primarily to preserve backwards compatibility with older integrations. An AI assistant may generate a technically correct refactoring that removes what appears to be redundant logic. To the model, the change looks like an improvement. To the maintainer, it introduces a future bug. The maintainer must therefore inspect, reject or rewrite the suggestion.
In this situation, expertise increases review effort because the expert recognises risks invisible to the AI. The generated code becomes another artefact requiring evaluation rather than a finished solution.
Why Familiar Repositories Leave Less Room for AI Gains
AI tools tend to create the greatest productivity gains when developers are uncertain about implementation details or need help generating boilerplate. Familiar repositories reduce both needs.
Experienced maintainers often already know:
- Which files require modification.
- Which architectural patterns should be followed.
- Which tests are likely to fail.
- Which implementation approaches have previously been rejected.
- Which maintainers will review the change. [Metr]metr.org2025 07 10 early 2025 ai experienced os dev studyMeasuring the Impact of Early-2025 AI on Experienced…10 Jul 2025 — We conduct a randomized controlled trial (RCT) to understand ho…
Because much of the discovery work is already complete, AI has fewer opportunities to contribute unique information. The expert frequently starts with a near-complete mental model of the solution.
This shifts the cost-benefit calculation. Instead of replacing expensive reasoning, the AI adds an additional artefact that must be inspected. The value of generated code declines while the review burden remains.
Axios summarised this distinction by noting that AI tools often perform better when creating new software than when extending complex, ongoing projects where existing structure and context matter more. [Axios]axios.comAI's elusive coding speedupExamining experienced developers working with large open-source projects, the research found that AI often slows down coders rather than…
The Illusion of Speed
One of the most intriguing findings is that developers often felt faster even when objective measurements showed a slowdown.
METR found that participants believed AI improved their productivity both before and after the experiment. Independent commentary on the study highlighted the gap between perceived and measured performance. [arXiv+2Sean Goedecke]arxiv.orgMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity…
Several factors may contribute to this perception:
- Watching code appear instantly creates a strong impression of progress.
- Editing generated code can feel easier than starting from a blank file.
- Cognitive effort may decrease even when elapsed time increases.
- Developers may remember rapid generation moments more vividly than lengthy verification work. [Reuters]reuters.comAI slows down some experienced software developers, study finds A new study by the AI research nonprofit METR found that using AI codingDespite initially believing AI would speed up their work by 24%, developers took 19% longer to complete tasks with AI assistance. The res…
In other words, AI can reduce the feeling of labour while increasing the amount of time required to finish a task. For maintainers, this distinction matters because software projects are ultimately constrained by completed work, not by subjective impressions of productivity.
What This Means for Repository Maintainers
The evidence does not suggest that AI coding assistants are universally harmful. Instead, it indicates that their usefulness depends heavily on context.
For experts working in repositories they know well, the central challenge is not generating code but preserving correctness, consistency and maintainability. AI-generated output frequently shifts effort from writing to reviewing. When project knowledge is extensive and quality standards are high, that review burden can outweigh the benefits of automated code generation. [Metr+2arXiv]metr.org2025 07 10 early 2025 ai experienced os dev studyMeasuring the Impact of Early-2025 AI on Experienced…10 Jul 2025 — We conduct a randomized controlled trial (RCT) to understand ho…
This helps explain why AI tools can simultaneously feel helpful and produce slower completion times. In familiar repositories, the expert’s deepest advantage is contextual understanding. The more important that understanding becomes, the more likely it is that AI-generated code will require careful scrutiny before it can be trusted. Recent research on AI-authored commits and AI-assisted review systems also points to ongoing concerns about maintenance costs and the need for substantial human quality assurance, reinforcing the idea that generation is only part of the development process. [arXiv]arxiv.orgDebt Behind the AI Boom: A Large-Scale Empirical Study of AI-Generated Code in the WildMarch 30, 2026…
Amazon book picks
Further Reading
Books and field guides related to Why Can AI Make Expert Developers Slower?. Use these as the next step if you want deeper reading beyond the article.
Designing Data-intensive Applications
Demonstrates the complexity of mature systems and repositories.
The Pragmatic Programmer
Rating: 4.5/5 from 7 Google Books ratings
Explains expert judgment and project-specific knowledge.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: metr.org
Title: 2025 07 10 early 2025 ai experienced os dev study
Link: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/Source snippet
Measuring the Impact of Early-2025 AI on Experienced...10 Jul 2025 — We conduct a randomized controlled trial (RCT) to understand ho...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2507.09089Source snippet
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity...
-
Source: reuters.com
Link: https://www.reuters.com/[business/aiSource snippet
Despite initially believing AI would speed up their work by 24%, developers took 19% longer to complete tasks with AI assistance. The res...
-
Source: techradar.com
Title: Tech Radar Using AI might actually slow down experienced devs
Link: https://www.techradar.com/pro/using-ai-might-actually-slow-down-experienced-devsSource snippet
Conducted on 16 seasoned developers working on 246 tasks across familiar open-source projects, the study revealed that while developers i...
-
Source: axios.com
Title: AI’s elusive coding speedup
Link: https://www.axios.com/2025/07/15/ai-coding-productivity-studySource snippet
Examining experienced developers working with large open-source projects, the research found that AI often slows down coders rather than...
-
Source: time.com
Title: In the Loop: AI Promised Faster Coding
Link: https://time.com/7302351/ai-software-coding-study/Source snippet
This Study DisagreesA recent METR study challenges the assumption that AI accelerates software development. In tests with 16 experienced...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2603.28592Source snippet
Debt Behind the AI Boom: A Large-Scale Empirical Study of AI-Generated Code in the WildMarch 30, 2026...
Published: March 30, 2026
-
Source: arxiv.org
Title: arXiv Does AI [Code Review]({{ ‘code-review/’ | relative_url }}) Lead to Code Changes? A Case Study of Git Hub Actions
Link: https://arxiv.org/abs/2508.18771 -
Source: metr.org
Link: https://metr.org/Source snippet
METR (pronounced 'meter') is a research nonprofit that scientifically measures whether and when AI systems might threaten catastrophi...
-
Source: metr.org
Link: https://metr.org/blog/2026-02-24-uplift-update/Source snippet
We are Changing our Developer Productivity Experiment...24 Feb 2026 — Our early 2025 study found the use of AI causes tasks to take 19%...
-
Source: arxiv.org
Link: https://arxiv.org/pdf/2507.09089Source snippet
Measuring the Impact of Early-2025 AI on Experienced...by J Becker · 2025 · Cited by 118 — We provide evidence that recent AI systems sl...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2510.10165Source snippet
Authors:...Read more...
-
Source: itpro.com
Title: IT Pro Think AI coding tools are speeding up work?
Link: https://www.itpro.com/software/development/think-ai-coding-tools-are-speeding-up-work-think-again-theyre-actually-slowing-developers-downSource snippet
Think again - they're actually slowing developers downA recent study by Model Evaluation & Threat Research (METR) challenges common assum...
-
Source: seangoedecke.com
Link: https://www.seangoedecke.com/impact-of-ai-study/Source snippet
Sean GoedeckeMETR's AI productivity study is really goodJul 11, 2025 — The other half is even more interesting: developers came away from...
Additional References
-
Source: reddit.com
Link: https://www.reddit.com/r/slatestarcodex/comments/1lwrb09/metr_finds_that_experienced_opensource_developers/Source snippet
METR finds that experienced open-source developers...METR finds that experienced open-source developers work 19% slower when using Early...
-
Source: linkedin.com
Link: https://www.linkedin.com/posts/thestevemitchell_measuring-the-impact-of-early-2025-ai-on-activity-7349786687230644225-OqTHSource snippet
METR study: AI assistants slow down experienced...A study from METR shows that experienced developers working with AI assistants produce...
-
Source: thezvi.substack.com
Link: https://thezvi.substack.com/p/on-metrs-ai-coding-rctSource snippet
METR's AI Coding RCTWe ran a randomized controlled trial to see how much AI coding tools speed up experienced open-source developers. The...
-
Source: comtrade360.com
Link: https://www.comtrade360.com/insights/ai-coding-assistants-are-great-for-mvps-and-demos-shipping-[productionSource snippet
AI coding assistants are great for MVPs and demos25 Feb 2026 — AI coding assistants are great for MVPs and demos – shipping production co...
-
Source: odsc.medium.com
Link: https://odsc.medium.com/new-study-finds-ai-tools-slow-experienced-developers-in-familiar-codebases-d92695f23665Source snippet
Study Finds AI Tools Slow Experienced Developers in...New Study Finds AI Tools Slow Experienced Developers in Familiar Codebases · Produ...
-
Source: linkedin.com
Link: https://www.linkedin.com/posts/luminousmen_measuring-the-impact-of-early-2025-ai-on-activity-7367262407888412672-Vk8ASource snippet
AI coding tools slow down experienced developers, but...Back in July 2025, a study from METR made some noise: “Early-2025 AI coding tool...
Published: July 2025
-
Source: linkedin.com
Title: ai coding assistants may slow developers 19 familiar codebases jha vfqvc
Link: https://www.linkedin.com/pulse/ai-coding-assistants-may-slow-developers-19-familiar-codebases-jha-vfqvcSource snippet
AI Coding Assistants May Slow Developers by 19% in...A July 2025 study by METR found that developers using Cursor in familiar codebases...
Published: July 2025
-
Source: britannica.com
Link: https://www.britannica.com/technology/artificial-intelligenceSource snippet
er or computer-controlled robot to perform tasks commonly associated with intelligent beings.Read more...
-
Source: actuia.com
Title: a metr study reveals that ai slows down experienced developers
Link: https://www.actuia.com/en/news/a-metr-study-reveals-that-ai-slows-down-experienced-developers/Source snippet
A METR Study Reveals that AI Slows Down Experienced...Jul 16, 2025 — TLDR: A study by METR found that experienced developers were 19% s...
-
Source: letsdatascience.com
Title: developers thought ai made them faster the data said otherwise
Link: https://letsdatascience.com/blog/developers-thought-ai-made-them-faster-the-data-said-otherwiseSource snippet
AI Coding Tools Made Developers 19% Slower: METR Study20 Feb 2026 — A gold-standard scientific study found AI coding tools made experienc...
Topic Tree