Does faster coding mean better programming?

Introduction

Coding assistants such as GitHub Copilot can make developers noticeably faster. The strongest controlled experiment found that programmers using Copilot completed a coding task about 56% faster than a control group. That result is real and important. However, faster completion is not the same thing as deeper understanding. A developer can successfully ship code, pass tests, and close a ticket while remaining unable to explain why the solution works, how it might fail, or how to modify it safely later.[arXiv]arxiv.orgarXiv The Impact of AI on Developer Productivity: Evidence from Git Hub CopilotThe Impact of AI on Developer Productivity: Evidence from GitHub CopilotFebruary 13, 2023…Published: February 13, 2023

This distinction matters because software engineering is not only about producing code. It also involves reasoning about behaviour, diagnosing failures, evaluating trade-offs, and maintaining systems over time. The central question is therefore not whether Copilot increases speed—it often does—but whether that speed sometimes masks shallow understanding.

Does faster coding mean better programming?

The most widely cited evidence for Copilot’s productivity benefits comes from a controlled GitHub study in which developers implemented an HTTP server in JavaScript. Participants with Copilot completed the task 55.8% faster than those without it.[arXiv]arxiv.orgarXiv The Impact of AI on Developer Productivity: Evidence from Git Hub CopilotThe Impact of AI on Developer Productivity: Evidence from GitHub CopilotFebruary 13, 2023…Published: February 13, 2023

What this study measured, however, was task completion speed under controlled conditions. It did not directly measure long-term comprehension, architectural judgement, debugging skill, or the ability to explain generated code. Those are separate capabilities.

This distinction becomes clearer when examining later research and field studies. Real-world software development involves understanding existing systems, collaborating with colleagues, reviewing code, and maintaining software months or years after it is written. Researchers studying Copilot in a large public-sector software organisation found that developers often reported feeling more productive, yet commit-based productivity metrics showed no statistically significant change after adoption. The researchers noted a gap between subjective impressions of productivity and measurable engineering output.[arXiv]arxiv.orgDeveloper Productivity With and Without GitHub Copilot: A Longitudinal Mixed-Methods Case StudySeptember 24, 2025…Published: September 24, 2025

The lesson is not that productivity gains are imaginary. Rather, speed measurements answer a narrower question than many people assume. They show that developers can often produce code more quickly, not that they have become better programmers in every relevant sense.

What controlled productivity studies actually measure

A common misunderstanding is to treat productivity studies as proof of increased expertise. Most studies do not measure expertise directly.

Typically, they evaluate outcomes such as:

These are valuable metrics, but they do not reveal whether a developer could:

The distinction is similar to using a calculator. Completing arithmetic faster does not necessarily demonstrate a stronger understanding of mathematics. Likewise, generating code faster does not automatically demonstrate stronger software engineering judgement.

Several researchers and industry observers have therefore argued that coding-assistant studies should not be interpreted as comprehensive measures of engineering competence.[third-bit.com]third-bit.comThe task was…Read more…

Why working code is not the same as understood code

A generated solution can appear convincing long before it is fully understood.

One reason is automation bias: people tend to trust outputs produced by automated systems, especially when those outputs look polished and plausible. Research examining programmers reviewing Copilot-generated code found that developers devoted less visual attention to AI-generated code than to human-written code, despite the need for careful inspection. The authors warned that programmers should be alert to complacency when reviewing generated output.[arXiv]arxiv.orgHow Readable is Model-generated Code? Examining Readability and Visual Inspection of GitHub CopilotAugust 31, 2022…Published: August 31, 2022

This creates a subtle failure mode:

The result may be functioning software accompanied by weak mental models. A programmer knows that the code works but cannot clearly explain why it works.

This becomes especially risky when the code handles concurrency, security, networking, database interactions, or unfamiliar libraries. In such situations, understanding the mechanism is often more important than obtaining a working snippet.

Evidence that comprehension can lag behind assistance

Recent research has begun examining understanding directly rather than treating task completion as a proxy.

A 2026 randomised controlled study reported that developers using AI coding assistance while learning unfamiliar programming libraries scored lower on comprehension assessments than those who worked without such assistance. The study found a divide between developers who used AI primarily for conceptual guidance and those who delegated substantial code generation. Participants who relied heavily on generated code demonstrated weaker understanding of the underlying material.[InfoQ]infoq.comai coding skill formationAnthropic Study: AI Coding Assistance Reduces Developer…23 Feb 2026 — Anthropic research shows developers using AI assistance sco…

Research on students has raised similar questions. Studies investigating Copilot’s effects in educational settings increasingly focus not only on whether students can complete assignments but also on whether they can explain their solutions and transfer knowledge to new problems.[ACM Digital Library]dl.acm.orgACM Digital LibraryThe Effects of GitHub Copilot on Computing Students…by MIH Shihab · 2025 · Cited by 37 — This paper investigates ho…

These findings support a broader concern: AI assistance may sometimes substitute for learning rather than reinforce it. When that happens, productivity gains and knowledge gains diverge.

The review burden hidden behind apparent speed

A second reason speed can conceal shallow understanding is that generated code often shifts work rather than eliminating it.

Writing code is only one part of development. Someone must still:

Studies and surveys increasingly describe a “verification burden” or “verification debt”: developers save time generating code but spend additional time checking whether it is trustworthy.[IT Pro]itpro.comDespite 96% of developers admitting they don't fully trust the functional correctness of AI-written code, many skip verification—primaril…

This becomes problematic when verification is skipped. Surveys have found that many developers do not consistently review AI-generated code despite expressing limited trust in its correctness. Other respondents reported that verifying AI-generated code can be more time-consuming than reviewing code written by colleagues because the reasoning behind the generated solution is often unclear.[TechRadar]techradar.comTech Radar Devs don't trust AI codeAlarmingly, less than half (48%) consistently check this code before committing it, opening the door to bugs and vulnerabilities. Even th…

In these situations, apparent productivity can be misleading. The speed comes from reducing visible effort at code creation while increasing hidden effort in validation and maintenance.

A revealing test: can the developer explain the code?

One practical way to distinguish genuine understanding from superficial acceptance is to ask developers to explain their AI-assisted work without looking at the generated output.

A developer who truly understands a Copilot suggestion should generally be able to explain:

If those questions cannot be answered, the developer may be acting more as an approver than as an author.

This does not mean the code is necessarily bad. It means the human understanding behind the code may be weaker than the productivity statistics suggest.

Questions developers should answer before accepting a suggestion

Before accepting an AI-generated implementation, developers can ask a few simple questions:

Could I explain this to another engineer?

If not, additional review is probably required.

What problem is each major section solving?

Understanding should exist at the level of intent, not just syntax.

What would break if requirements changed?

Maintenance requires more than knowing that the current version works.

What assumptions does this code make?

Generated solutions often rely on unstated assumptions about inputs, environments, or libraries.

How would I test failure cases?

Understanding becomes visible when reasoning about things going wrong.

Could I reproduce the solution without the assistant?

The answer need not be yes every time, but the question reveals whether the tool is supporting learning or replacing it.

Why this matters for understanding artificial intelligence

The Copilot debate illustrates a broader lesson about AI systems. Artificial intelligence can increase output without proportionally increasing human understanding. A person may become faster, more productive, and more successful at completing tasks while simultaneously relying on reasoning they do not fully possess themselves.[arXiv]arxiv.orgarXiv The Impact of AI on Developer Productivity: Evidence from Git Hub CopilotThe Impact of AI on Developer Productivity: Evidence from GitHub CopilotFebruary 13, 2023…Published: February 13, 2023

That possibility does not negate the value of coding assistants. It simply changes how their benefits should be interpreted. Speed is a useful measure of performance, but it is an incomplete measure of expertise. When evaluating AI-assisted work, the crucial question is not only how quickly a task was completed, but whether the human involved can still explain, maintain, and improve the result after the assistant is gone.

Amazon book picks

Further Reading

Books and field guides related to Does faster coding mean better programming?. Use these as the next step if you want deeper reading beyond the article.

Book

The Pragmatic Programmer

By Andrew Hunt, David Thomas

Rating: 4.5/5 from 7 Google Books ratings

Separates productivity from true engineering skill.

See on Amazon

Book

Clean Code

By Robert C. Martin

Focuses on understanding and maintaining code quality.

See on Amazon

Book

Co-Intelligence

By Ethan Mollick

Provides context for AI productivity tools and human oversight.

See on Amazon

Book

Code Complete

By Steve McConnell

Explores what effective programming actually involves.

See on Amazon

Browse more on Amazon: The Pragmatic Programmer Clean Code Co Intelligence

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Example marketplace items related to this page. Use the search link to explore similar finds on eBay.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Example eBay listing

50 Pcs/Pack Programmers Hackers Linux Python Stickers For Laptop PC Sticker Bomb

SearcheBay.co.uk: programming sticker pack

Browse similar oneBay.co.uk

Example eBay listing

Soviet vintage space stickers — Sticker sheet, sticker pack, macbook skin

SearcheBay.co.uk: programming sticker pack

Browse similar oneBay.co.uk

Example eBay listing

Coding Symbol Vinyl Decal | Programming Software Development | Die Cut Sticker

SearcheBay.co.uk: programming sticker pack

Browse similar oneBay.co.uk

Example eBay listing

Antarctica Sticker Pack - US Antarctic Program & Search and Rescue Decals Lot

SearcheBay.co.uk: programming sticker pack

Browse similar oneBay.co.uk

Browse more oneBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: arxiv.org
Title: arXiv The Impact of AI on Developer Productivity: Evidence from Git Hub Copilot
Link:https://arxiv.org/abs/2302.06590

Published: February 13, 2023

2. Source: arxiv.org
Link:https://arxiv.org/pdf/2302.06590

3. Source: arxiv.org
Link:https://arxiv.org/abs/2509.20353

Published: September 24, 2025

4. Source: third-bit.com
Link:https://third-bit.com/2026/05/20/twelve-ways-to-be-wrong/

5. Source: arxiv.org
Link:https://arxiv.org/html/2602.03593v1

6. Source: arxiv.org
Link:https://arxiv.org/abs/2208.14613

Published: August 31, 2022

7. Source: infoq.com
Title: ai coding skill formation
Link:https://www.infoq.com/news/2026/02/ai-coding-skill-formation/

8. Source: dl.acm.org
Link:https://dl.acm.org/doi/full/10.1145/3702652.3744219

9. Source: techradar.com
Title: Tech Radar Devs don’t trust AI code
Link:https://www.techradar.com/pro/devs-dont-trust-ai-code-but-many-say-they-still-dont-check-it-anyways

10. Source: github.com
Title: Git Hub · Change is constant
Link:https://github.com/

11. Source: github.com
Link:https://github.com/features/copilot

12. Source: docs.github.com
Title: [code review]({{ ‘code-review/’ | relative_url }})
Link:https://docs.github.com/en/copilot/concepts/agents/code-review

13. Source: learn.github.com
Link:https://learn.github.com/learning-pathways/github-copilot

14. Source: arxiv.org
Link:https://arxiv.org/pdf/2302.00438

15. Source: arxiv.org
Link:https://arxiv.org/html/2504.16485v1

16. Source: github.blog
Link:https://github.blog/

17. Source: github.blog
Title: Does Git Hub Copilot improve code quality?
Link:https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/

18. Source: github.blog
Title: survey reveals ais impact on the developer experience
Link:https://github.blog/news-insights/research/survey-reveals-ais-impact-on-the-developer-experience/

19. Source: cacm.acm.org
Title: measuring github copilots impact on productivity
Link:https://cacm.acm.org/research/measuring-github-copilots-impact-on-productivity/

20. Source: dl.acm.org
Link:https://dl.acm.org/doi/full/10.1145/3709353

21. Source: itpro.com
Link:https://www.itpro.com/software/development/software-developers-not-checking-ai-generated-code-verification-debt

Additional References

22. Source: linkedin.com
Link:https://www.linkedin.com/posts/aagupta_41-of-all-code-shipped-in-2025-was-ai-generated-activity-7438810992651743232-II6u

23. Source: linkedin.com
Link:https://www.linkedin.com/posts/drelleobrien_more-code-less-validation-risk-factors-activity-7413966333069131776-Yfj5

24. Source: cut-the-saas.com
Link:https://cut-the-saas.com/ai/[generative-ai

25. Source: xebia.com
Link:https://xebia.com/blog/github-copilot-productivity/

26. Source: medium.com
Link:https://medium.com/%40reliabledataengineering/ai-is-writing-46-of-all-code-github-copilots-real-impact-on-15-million-developers-787d789fcfdc

27. Source: git-scm.com
Link:https://git-scm.com/

28. Source: zenml.io
Link:https://www.zenml.io/llmops-database/improving-contextual-understanding-in-github-copilot-through-advanced-prompt-engineering

29. Source: linkedin.com
Link:https://www.linkedin.com/posts/sanjoyghosh82_transforming-software-development-evaluating-activity-7213884338935918594-oNGg

30. Source: researchgate.net
Link:https://www.researchgate.net/publication/368473822_The_Impact_of_AI_on_Developer_Productivity_Evidence_from_GitHub_Copilot

31. Source: ksred.com
Title: ai for coding why most developers are getting it wrong and how to get it right
Link:https://www.ksred.com/ai-for-coding-why-most-developers-are-getting-it-wrong-and-how-to-get-it-right/

Published: July 2025