A development team at a mid-sized software company used an AI coding assistant to patch a known vulnerability in their authentication library. The AI generated clean, functional code that passed all unit tests, and the team merged it into production.
Three weeks later, their security scanner flagged the same CVE they thought they'd fixed. The AI had reintroduced the exact vulnerability that human developers had already resolved.
This isn't an isolated incident. Research commissioned by Checkmarx shows that 65-75% of AI-generated patches resurrect vulnerabilities already fixed by humans. AI-generated code contributed to 25% of breaches in the study period, making it a top-four breach vector. The bigger problem is how teams treat AI output as trustworthy by default.
What Happened
The team used an AI coding assistant to refactor authentication logic in a Python web application. The assistant generated code that:
- Compiled without errors
- Passed the existing test suite
- Implemented the requested functionality
- Reintroduced a previously patched SQL injection vulnerability (CVE-2023-29159)
The code review focused on functional correctness. The security team's static analysis tools flagged the vulnerability during the next scheduled scan, but by then the code had been in production for three weeks.
Timeline
Week 1, Day 1: Developer uses AI assistant to refactor authentication module
Week 1, Day 2: Code review approves changes based on functional tests
Week 1, Day 3: Deployment to production
Week 3, Day 5: Scheduled security scan flags CVE-2023-29159
Week 3, Day 6: Team confirms the vulnerability matches one fixed six months earlier
Week 3, Day 7: Emergency patch and rollback
Which Controls Failed
The team had three control gaps:
No pre-commit security scanning for AI-generated code. The CI/CD pipeline ran security scans weekly, not on every commit. AI output bypassed the same scrutiny applied to vendor libraries or contractor code.
Code review checklist didn't account for AI sources. Reviewers verified that the code worked but didn't check whether it matched known vulnerability patterns. The team treated AI output like code from a senior developer, not like code from an unknown source.
No comparison against previous fixes. The team had patched this exact vulnerability before but had no automated way to detect when the same pattern reappeared. Their Git history contained the fix, but no one thought to check whether the AI's approach matched the vulnerable version or the patched version.
What Standards Require
PCI DSS v4.0.1 Requirement 6.3.2 mandates that custom software be reviewed before release to production to identify and correct coding vulnerabilities. This applies regardless of code source. AI-generated code is custom software, and "review" means security review, not just functional validation.
OWASP ASVS v4.0.3, Section V1.14.1 requires that the build pipeline contain an automated security analysis step that breaks the build if high-severity issues are found. The control doesn't exempt AI-generated code. If your SAST tools run weekly instead of on every commit, you're not meeting this requirement when you merge AI output between scans.
ISO 27001 Control 8.25 (secure coding) states that secure coding principles shall be applied to software development. The control requires you to define what "secure coding" means in your environment and verify compliance. If your definition doesn't address AI-generated code, you have a gap.
NIST 800-53 Rev 5 Control SA-11 (developer testing and evaluation) requires security testing during development, not just after deployment. When you use AI to generate code, you're outsourcing development to a tool that doesn't perform security testing. The control still applies to you.
Lessons and Actions
Research by Ilya Kabanov at The Weather Report found that functional success in AI-generated code reached 83-95% across models, but secure-and-functional success dropped to 24-36%. Your AI assistant is optimized for "does it work?" not "is it safe?" Treat every AI-generated block as untrusted input.
Action 1: Run SAST on every commit that includes AI-generated code. Don't wait for your weekly scan. If you're using GitHub Copilot, Claude, or similar tools, configure your pipeline to treat those commits as high-risk and trigger immediate analysis.
Action 2: Build a vulnerability pattern library from your Git history. Extract every CVE fix you've committed in the past two years. Create signatures or rules that flag when similar patterns reappear. This doesn't require expensive tools; a script that runs git diff against known vulnerable patterns will catch obvious regressions.
Action 3: Update your code review checklist. Add a mandatory question: "If this code was AI-generated, did you verify it doesn't match any previously patched vulnerability?" Train reviewers to check the blame history for the files being modified.
Action 4: Prompt engineering won't save you. The research showed that telling AI to "follow best security practices" didn't meaningfully improve secure code generation. Don't rely on better prompts. Rely on verification.
Action 5: Document AI usage in your SDLC documentation. If you're audited for PCI DSS, ISO 27001, or SOC 2 Type II, the auditor will ask how you ensure third-party and tool-generated code meets security requirements. "We trust the AI" isn't an acceptable answer. "We scan all AI output before merge and compare against known vulnerability patterns" is.
Your AI coding assistant doesn't know which vulnerabilities you've already fixed. It trained on code that includes both vulnerable and patched versions, and it can't reliably distinguish between them. Until that changes, treat AI-generated code the way you'd treat code from an intern who's never seen your codebase: functional, possibly correct, and definitely requiring security review before it ships.
