Within Code Assistants
Can you trust code you did not write?
Plausible AI-generated code can still contain flaws, so the real bottleneck often shifts from writing code to reviewing it well.
On this page
- Why polished generated code can feel safer than it is
- Security questions every AI assisted developer should ask
- How senior engineers absorb review and rework burdens
Page outline Jump by section
Introduction
AI coding assistants can dramatically reduce the time needed to produce software, but they also change where the difficult work happens. The bottleneck often shifts from writing code to reviewing it. A developer can now generate hundreds of lines of plausible, well-formatted code in seconds. The challenge is determining whether that code is secure, maintainable, and appropriate for the system into which it is being introduced.
This creates a subtle productivity-understanding trade-off. Code that looks professional can create a false sense of confidence. Security researchers, software engineering studies, and industry reports repeatedly find that AI-generated code can contain vulnerabilities even when it passes functional tests and appears production-ready. As a result, the value of human expertise increasingly lies not in typing code but in questioning, validating, and revising it. [NYU Center for Cyber Security+2Veracode]cyber.nyu.eduNYU Center for Cyber SecurityCCS researchers find Github CoPilot generates vulnerable…Oct 15, 2021 — CCS researchers find Github CoPil…
Can You Trust Code You Did Not Write?
The short answer is: not automatically.
Traditional programming already requires code review because humans make mistakes. AI-generated code introduces an additional challenge. Large language models are trained to predict likely code patterns, not to guarantee security. They can produce convincing implementations that satisfy a prompt while quietly omitting safeguards, using unsafe defaults, or reproducing insecure examples that appeared in training data. [SonarSource+2SecureFlag]sonarsource.comSonar Source OWASP LLM Top 10: How it Applies to Code GenerationOWASP LLM Top 10: How it Applies to Code GenerationOctober 21, 2025 — AI-generated code introduces unique risks because develo…
Multiple studies have documented this problem. A widely discussed early study from researchers at New York University found that around 40% of generated solutions contained vulnerabilities or exploitable flaws. Subsequent research examining real-world repositories and production workflows has continued to identify significant rates of security weaknesses in AI-generated code. [NYU Center for Cyber Security]cyber.nyu.eduNYU Center for Cyber SecurityCCS researchers find Github CoPilot generates vulnerable…Oct 15, 2021 — CCS researchers find Github CoPil…
The important lesson is that functional correctness and security are different questions. Code may compile, pass tests, and deliver the requested feature while still exposing systems to injection attacks, weak authentication, insecure randomness, path traversal vulnerabilities, or unsafe handling of sensitive data. [arXiv+2arXiv]arxiv.orgOpen source on arxiv.org.
Why Polished Generated Code Can Feel Safer Than It Is
One of the most significant risks is psychological rather than technical.
Human reviewers often use visual cues when assessing software quality. Clear naming, consistent formatting, comments, and coherent structure can create an impression of competence. Modern coding assistants are exceptionally good at producing these signals.
The result is a form of automation bias: people become more likely to trust a system because its output appears sophisticated. Security researchers have repeatedly observed that AI-generated code may contain vulnerabilities hidden beneath professional-looking structure. Veracode’s large-scale testing of more than 100 models found that many generated solutions still introduced common security weaknesses despite appearing production-ready. [Veracode+2TechRadar]veracode.comgenai code security reportInsights from 2025 GenAI Code Security Report30 Jul 2025 — How secure is code generated by AI? We asked 100+ AI models to write c…
Several factors make this problem worse:
- Volume: AI can generate much more code than a human would typically write in the same period, increasing review workload.
- Speed pressure: Teams may feel pressure to accept working code quickly because productivity metrics appear positive.
- Authority effects: Developers may assume advanced AI systems are applying best practices automatically.
- Fragmented understanding: A programmer may understand the prompt they wrote but not every implementation detail in the resulting output.
Research examining AI-assisted development workflows suggests that security thinking is increasingly being pushed downstream into review activities rather than embedded in the initial act of coding. [arXiv]arxiv.orgHow AI Coding Assistants Transform Developers' Security…5 days ago — We find that AI coding assistants reorganize rather than eli…
Security Questions Every AI-Assisted Developer Should Ask
Security review after AI-generated code requires a different mindset from ordinary code inspection. The reviewer should assume that the code may contain hidden risks regardless of how polished it appears.
Useful review questions include:
How does the code handle untrusted input?
Many vulnerabilities arise when external input reaches databases, file systems, command interpreters, or browsers without proper validation or encoding. Reviewers should trace the entire data flow rather than relying on the generated implementation’s apparent correctness. [arXiv]arxiv.orgThe Hidden Risks of LLM-Generated Web Application Code29 Apr 2025 — The analysis reveals critical vulnerabilities in authentication…
What security assumptions are being made?
Generated code often omits explicit discussion of trust boundaries, authentication requirements, authorisation checks, or threat models. The absence of these considerations can be more revealing than the presence of the code itself. [SonarSource]sonarsource.comSonar Source OWASP LLM Top 10: How it Applies to Code GenerationOWASP LLM Top 10: How it Applies to Code GenerationOctober 21, 2025 — AI-generated code introduces unique risks because develo…
Are dependencies and libraries appropriate?
AI assistants sometimes recommend outdated packages, insecure examples, or dependency combinations that introduce supply-chain risk. The generated code may work while relying on components that should not be used in production. [SecureFlag]blog.secureflag.comthe risks of generative ai coding in software developmentThe risks of generative AI coding in software development16 Oct 2024 — One of the most noticeable risks with AI-generated code…
What happens when something fails?
Error handling often receives less attention than primary functionality. Reviewers should examine exception handling, logging, timeout behaviour, rate limiting, and resource management. These areas frequently contain security weaknesses. [GrowExx]growexx.comai generated code owasp top 10AI-Generated Code OWASP Top 102 Apr 2026 — When AI coding tools generate LLM integration code, they commonly omit: rate limiting o…
Has the code been analysed independently?
Static analysis tools, dependency scanners, security linters, and automated testing become more important in AI-assisted workflows because they provide an independent check on generated output. Research consistently recommends combining AI-assisted development with automated verification rather than treating generated code as trusted by default. [arXiv+2arXiv]arxiv.orgOpen source on arxiv.org.
How Senior Engineers Absorb Review and Rework Burdens
A common misconception is that AI coding assistants primarily reduce engineering effort. In many organisations they redistribute it.
Junior developers can now generate substantial features rapidly, but the responsibility for evaluating those features often falls on more experienced engineers. Senior reviewers must verify architectural consistency, security assumptions, maintainability, and operational risks that may not be visible in the generated code itself. [arXiv]arxiv.orgHow AI Coding Assistants Transform Developers' Security…5 days ago — We find that AI coding assistants reorganize rather than eli…
This creates several new burdens:
Reviewing More Code Than Before
If one engineer can generate three times as much code, review capacity does not automatically increase by the same amount. Security review can become the limiting factor in delivery pipelines. Industry surveys and practitioner reports increasingly describe review bottlenecks as a hidden cost of AI-assisted development. [TechRadar]techradar.comTech Radar Nearly all security bosses are worried about AI safetyAn overwhelming 90% of security leaders report active concerns about AI safety, particularly as AI coding tools become more widespread in…
Reconstructing Missing Reasoning
Human-written code often carries traces of the author’s thinking. Reviewers can ask why a decision was made. With generated code, the reasoning may be absent, inconsistent, or difficult to reconstruct.
Senior engineers may therefore spend time reverse-engineering the logic behind an implementation before they can evaluate whether it is safe. This shifts effort from creation to interpretation. [Medium]medium.comTrue Cost of AI-Generated CodeAI-assisted programming shifts the mental burden from code generation to the more demanding task of c…
Managing Security Debt
Unchecked AI-generated code can accumulate hidden vulnerabilities over time. Research examining thousands of AI-attributed files in public repositories identified thousands of weakness instances across dozens of Common Weakness Enumeration categories. The concern is not a single catastrophic flaw but the gradual growth of security debt throughout a codebase. [arXiv]arxiv.orgarXiv[2510.26103] Security Vulnerabilities in AI-Generated CodeOctober 30, 2025 — by M Schreiber · 2025 · Cited by 12 — This paper presen…
Security Review as a Policy Response
Because the risk is systematic rather than accidental, many organisations increasingly treat AI-generated code as a governance issue.
Common policy approaches include:
- Mandatory human review before deployment.
- Security scanning integrated into continuous integration pipelines.
- Additional review requirements for authentication, encryption, payments, and other high-risk components.
- Documentation of where AI-generated code was used.
- Treating AI-generated code similarly to third-party code that must be verified before adoption. [TechRadar+2Veracode]techradar.comTech Radar Nearly all security bosses are worried about AI safetyAn overwhelming 90% of security leaders report active concerns about AI safety, particularly as AI coding tools become more widespread in…
This approach recognises a key reality: AI systems can accelerate software production, but they do not eliminate the need for judgement. If anything, the ability to generate code rapidly increases the importance of careful review because mistakes can be produced and propagated at greater speed.
The Real Skill Shift
The most significant change may be educational rather than technical.
Historically, programming expertise was closely tied to the ability to construct code from scratch. In an AI-assisted environment, expertise increasingly includes the ability to evaluate generated solutions, identify hidden assumptions, recognise insecure patterns, and decide when a seemingly correct implementation should be rejected.
Recent research characterises this as a shift from preventive security to reactive security: developers spend less effort producing code and more effort assessing whether generated code can be trusted. [arXiv]arxiv.orgHow AI Coding Assistants Transform Developers' Security…5 days ago — We find that AI coding assistants reorganize rather than eli…
Understanding artificial intelligence therefore requires understanding a broader lesson. Automation does not necessarily remove difficult work. Often it moves that work elsewhere. In AI-assisted software development, the challenge is no longer simply writing code. It is knowing when not to trust code that writes itself.
Amazon book picks
Further Reading
Books and field guides related to Can you trust code you did not write?. Use these as the next step if you want deeper reading beyond the article.
The Web Application Hacker's Handbook
Shows how seemingly functional code can still be vulnerable.
Software Security
Rating: 4.0/5 from 5 Google Books ratings
Focuses on finding and preventing vulnerabilities.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: cyber.nyu.edu
Link: https://cyber.nyu.edu/2021/10/15/ccs-researchers-find-github-copilot-generates-vulnerable-code-40-of-the-time/Source snippet
NYU Center for Cyber SecurityCCS researchers find Github CoPilot generates vulnerable...Oct 15, 2021 — CCS researchers find Github CoPil...
-
Source: veracode.com
Title: genai code security report
Link: https://www.veracode.com/blog/genai-code-security-report/Source snippet
Insights from 2025 GenAI Code Security Report30 Jul 2025 — How secure is code generated by AI? We asked 100+ AI models to write c...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2508.14727 -
Source: sonarsource.com
Title: Sonar Source OWASP LLM Top 10: How it Applies to Code Generation
Link: https://www.sonarsource.com/resources/library/owasp-llm-code-generation/Source snippet
OWASP LLM Top 10: How it Applies to Code GenerationOctober 21, 2025 — AI-generated code introduces unique risks because develo...
Published: October 21, 2025
-
Source: blog.secureflag.com
Title: the risks of [generative ai]({{ ‘generative-ai/’ | relative_url }}) coding in software development
Link: https://blog.secureflag.com/2024/10/16/the-risks-of-generative-ai-coding-in-software-development/Source snippet
The risks of generative AI coding in software development16 Oct 2024 — One of the most noticeable risks with AI-generated code...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2310.02059 -
Source: arxiv.org
Link: https://arxiv.org/html/2504.20612v1Source snippet
The Hidden Risks of LLM-Generated Web Application Code29 Apr 2025 — The analysis reveals critical vulnerabilities in authentication...
-
Source: techradar.com
Link: https://www.techradar.com/pro/nearly-half-of-all-code-generated-by-ai-found-to-contain-security-flaws-even-big-llms-affectedSource snippet
The research analyzed over 100 large language models (LLMs) across 80 coding tasks and revealed no significant improvement in security pe...
-
Source: arxiv.org
Link: https://arxiv.org/html/2605.23130v2Source snippet
How AI Coding Assistants Transform Developers' Security...5 days ago — We find that AI coding assistants reorganize rather than eli...
-
Source: medium.com
Link: https://medium.com/%40justhamade/true-cost-of-ai-generated-code-f4362391790cSource snippet
True Cost of AI-Generated CodeAI-assisted programming shifts the mental burden from code generation to the more demanding task of c...
-
Source: techradar.com
Title: Tech Radar Why LLMs are plateauing
Link: https://www.techradar.com/pro/why-llms-are-plateauing-and-what-that-means-for-software-securitySource snippet
While LLMs like OpenAI's GPT-5 have shown improved accuracy in producing secure code due to enhanced reasoning capabilities, most models—...
-
Source: growexx.com
Title: ai generated code owasp top 10
Link: https://www.growexx.com/blog/ai-generated-code-owasp-top-10/Source snippet
AI-Generated Code OWASP Top 102 Apr 2026 — When AI coding tools generate LLM integration code, they commonly omit: rate limiting o...
-
Source: arxiv.org
Link: https://arxiv.org/pdf/2508.14727Source snippet
Assessing the Quality and Security of AI-Generated Codeby A Sabra · 2025 · Cited by 9 — This study argues that the common error patterns...
-
Source: techradar.com
Title: Tech Radar Nearly all security bosses are worried about AI safety
Link: https://www.techradar.com/pro/security/nearly-all-security-bosses-are-worried-about-ai-safety-with-a-third-saying-they-still-rely-on-manually-reviewing-code-before-launchSource snippet
An overwhelming 90% of security leaders report active concerns about AI safety, particularly as AI coding tools become more widespread in...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2510.26103Source snippet
arXiv[2510.26103] Security Vulnerabilities in AI-Generated CodeOctober 30, 2025 — by M Schreiber · 2025 · Cited by 12 — This paper presen...
Published: October 30, 2025
-
Source: arxiv.org
Title: arXiv Security of LLM-generated Code: A Comparative Analysis
Link: https://arxiv.org/abs/2605.23091 -
Source: medium.com
Link: https://medium.com/%40adnanmasood/security-analysis-and-validation-of-generative-ai-produced-code-d4218078bd63Source snippet
injection, XSS) to memory safety bugs (buffer overflows...
-
Source: medium.com
Link: https://medium.com/%40michael.hannecke/ai-is-writing-your-code-whos-[checking -
Source: owasp.org
Link: https://owasp.org/www-project-top-10-for-large-language-model-applications/Source snippet
OWASP FoundationOWASP Top 10 for Large Language Model ApplicationsLLM outputs may lead to downstream security exploits, including code ex...
-
Source: github.com
Title: Git Hub · Change is constant
Link: https://github.com/Source snippet
GitHub keeps you ahead. · GitHubFrom your first line of code to final deployment, GitHub provides AI and automation tools to help you bui...
-
Source: github.com
Title: 0x92 Appendix C AI for Code Generation.md
Link: https://github.com/OWASP/AISVS/blob/main/1.0/en/0x92-Appendix-C_AI_for_Code_Generation.mdSource snippet
Appendix C: AI-Assisted Secure Coding5 Jun 2026 — OWASP LLM Top 10 (2025) AI-generated code always goes through code review by a qualifie...
-
Source: genai.owasp.org
Link: https://genai.owasp.org/Source snippet
Gen AI Security Project: HomeOWASP's AI Security Solutions Landscape is a landmark guide for security professionals. It outlines key risk...
-
Source: github.dev
Title: Git Hub.dev Setting up your web editor
Link: https://github.dev/Source snippet
Setting up your web editor...
-
Source: github.blog
Link: https://github.blog/ -
Source: x.com
Title: Git Hub (@github) / Posts / XGit Hub (@github)
Link: https://x.com/githubSource snippet
GitHub (@github) / Posts / XGitHub (@github) - Posts - The AI-powered developer platform to build, scale, and deliver secure software. |...
Additional References
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/397089244_Security_Vulnerabilities_in_AI-Generated_Code_A_Large-Scale_Analysis_of_Public_GitHub_RepositoriesSource snippet
A Large-Scale Analysis of Public GitHub RepositoriesOct 30, 2025 — This paper presents a comprehensive empirical analysis of security vul...
-
Source: langprotect.com
Link: https://www.langprotect.com/ai-code-protectionSource snippet
AI Code Assistant Security & IDE Governance Platform91% increase in the manual review burden for senior developers. AI-generated code dif...
-
Source: softwareseni.com
Link: https://www.softwareseni.com/ai-generated-code-security-risks-why-vulnerabilities-increase-2-74x-and-how-to-prevent-them/Source snippet
Why Vulnerabilities Increase 2.74x and How to Prevent Them17 Feb 2026 — Here, we break down the actual security risks, look at real incid...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=nGxpctRd2OQSource snippet
AI Generated Code Has a Security ProblemThe real issue is developer laziness and misuse of the tools. Most people I see using AI rely on...
-
Source: researchgate.net
Title: Assessing the Security of Git Hub Copilot’s Generated Code Pearce et al
Link: https://www.researchgate.net/publication/382321925Assessing_the_Security_of_GitHub_Copilot%27s_Generated_Code-_A_Targeted_Replication_StudySource snippet
[49] demonstrated that over 40% of GitHub Copilot (one of the most widely used code generation tools) code suggestions contained vulnerab...
-
Source: ardura.consulting
Link: https://ardura.consulting/blog/ai-generated-code-why-45-percent-copilot-code-contains-security-vulnerabilities/Source snippet
AI Generated Code Security 2026 | Blog - ARDURA ConsultingDec 7, 2025 — 45% of AI code contains security vulnerabilities...
-
Source: endorlabs.com
Title: the most common security vulnerabilities in ai generated code
Link: https://www.endorlabs.com/learn/the-most-common-security-vulnerabilities-in-ai-generated-codeSource snippet
The Most Common Security Vulnerabilities in AI-Generated...12 Aug 2025 — Recent academic studies confirm that missing input sanitization...
-
Source: labs.cloudsecurityalliance.org
Title: csa research note ai generated code vulnerability surge 2026
Link: https://labs.cloudsecurityalliance.org/research/csa-research-note-ai-generated-code-vulnerability-surge-2026/Source snippet
Coding's Security Debt: The AI-Generated CVE SurgeVeracode tested over 100 large language models on security-sensitive coding tasks and f...
-
Source: softwareseni.com
Title: security risks in ai generated code and how to mitigate them
Link: https://www.softwareseni.com/security-risks-in-ai-generated-code-and-how-to-mitigate-them/Source snippet
Security Risks in AI-Generated Code and How to Mitigate Them28 Jan 2026 — Here's the reality: AI-generated code exhibits a 45% security v...
-
Source: endorlabs.com
Link: https://www.endorlabs.com/learn/llm-development-risksSource snippet
AI coding assistants can suggest code with common vulnerability patterns —buffer overflows...
Topic Tree