Scope - What This Guide Covers
This guide focuses on verification frameworks for AI-generated code in production environments. You'll find specific controls for validating outputs from large language models (LLMs), integration points with existing security tools, and measurable criteria for determining when AI-generated code meets your security standards.
What's included:
- Verification checkpoints mapped to common security standards
- Automation strategies for static and dynamic analysis
- Risk classification for different AI output types
- Integration patterns with CI/CD pipelines
What's not covered:
- Prompt engineering techniques
- AI model selection criteria
- General code review processes unrelated to AI outputs
Key Concepts and Definitions
AI-Generated Code: Source code, configuration files, or infrastructure-as-code produced by LLMs with minimal or no human modification before commit.
Verification Toil: Manual review time spent validating AI outputs. Data shows developers spend 24% of their work week on this, with 96% reporting they don't fully trust AI-generated code without manual intervention.
Trust Boundary: The point at which code transitions from AI-generated to production-ready. Your verification framework defines the controls at this boundary.
Impact Metrics: Measurements focused on security outcomes, such as vulnerabilities prevented and compliance gaps closed.
Requirements Breakdown
PCI DSS v4.0.1 Considerations
Requirement 6.2.4: Requires methods to prevent or mitigate common software attacks. AI-generated code must pass the same scrutiny as human-written code.
Requirement 6.3.2: Mandates security testing during development. Your framework must include automated security testing for all AI outputs before merge.
Requirement 11.3.1.1: Requires internal vulnerability scans. AI-generated infrastructure code needs the same scanning coverage as manually created configurations.
OWASP ASVS v4.0.3 Mapping
V1.14.2: Build pipelines must warn on out-of-date or insecure dependencies. AI tools frequently suggest deprecated libraries—your verification must catch these.
V5.1.1: Input validation requirements apply to AI-generated validation logic. Don't assume the LLM correctly implements allow-lists or sanitization.
V14.2.3: Dependencies must be checked for known vulnerabilities. AI suggestions often pull from outdated training data.
SOC 2 Type II Controls
CC6.1 (Logical and Physical Access Controls): AI-generated authentication logic requires manual review by a senior engineer. Automated tests alone are insufficient for access control code.
CC7.2 (System Monitoring): Your verification framework itself needs monitoring. Track false negative rates where AI code introduced vulnerabilities that passed initial checks.
Implementation Guidance
Stage 1: Classification
Before verification begins, classify the AI output:
Low-risk: Documentation, test data generation, boilerplate code
- Automated SAST + peer review
- 15-minute verification budget
Medium-risk: Business logic, API integrations, data transformations
- Automated SAST + DAST + senior engineer review
- 45-minute verification budget
- Requires security champion sign-off
High-risk: Authentication, authorization, cryptography, payment processing, data access layers
- Full security review including threat modeling
- Manual code review by two senior engineers
- Penetration testing for new attack surfaces
- No time budget—verification takes as long as needed
Stage 2: Automated Verification
Your pipeline must include:
Static Analysis:
- SAST tools configured with rules for your language stack
- Dependency vulnerability scanning (Snyk, Dependabot, or equivalent)
- Secret detection (GitGuardian, TruffleHog)
- License compliance checking
Policy-as-Code:
- OPA or similar for infrastructure code
- Custom rules for your organization's security patterns
- Validation that AI hasn't introduced anti-patterns you've previously banned
Configuration Validation:
- Security misconfigurations in cloud resources
- Overly permissive IAM roles
- Exposed endpoints or storage buckets
Stage 3: Manual Review Triggers
Automated checks should escalate to manual review when:
- AI suggests deprecated functions or libraries
- Code touches authentication or authorization boundaries
- External API calls are introduced
- Database queries are modified
- Cryptographic operations are implemented
- Environment variables or secrets are referenced
Stage 4: Continuous Monitoring
Post-deployment, track:
- Runtime errors in AI-generated code vs. human-written code
- Security incidents traced to AI outputs
- Performance degradation from inefficient AI suggestions
- Rollback frequency by code source
Common Pitfalls
Treating all AI output equally: A docstring and a password validation function require different verification intensity. Classification prevents both under-checking critical code and over-checking trivial changes.
Assuming the AI "knows" your security standards: LLMs don't have context on your organization's specific security policies, approved libraries, or architectural patterns. They generate plausible code, not compliant code.
Verification theater: Running tools without acting on findings creates false confidence. If your automated checks flag 47 issues but developers merge anyway, you don't have a verification framework—you have security theater.
Ignoring the toil metric: If verification takes longer than writing the code manually, your framework needs adjustment. The goal is trust with efficiency, not perfect security at infinite cost.
No feedback loop: When AI-generated code causes production issues, that information must flow back to your verification criteria. Update your automated checks and review triggers based on actual failures.
Speed-focused metrics: Lines of code generated per day tells you nothing about security posture. Track vulnerabilities prevented, compliance gaps avoided, and incident reduction instead.
Quick Reference Table
| Code Type | Risk Level | Automated Checks | Manual Review | Approval Required |
|---|---|---|---|---|
| Documentation, comments | Low | Linting only | Optional | Peer review |
| Test data generation | Low | Format validation | Optional | Peer review |
| Boilerplate (getters/setters) | Low | SAST | Optional | Peer review |
| Business logic | Medium | SAST + DAST | Required | Senior engineer |
| API integrations | Medium | SAST + DAST + dependency scan | Required | Senior engineer |
| Database queries | Medium | SAST + SQL injection tests | Required | Senior engineer + DBA |
| Authentication logic | High | Full suite + threat model | Two senior engineers | Security team |
| Authorization checks | High | Full suite + threat model | Two senior engineers | Security team |
| Cryptographic operations | High | Full suite + crypto review | Two senior engineers | Security team |
| Payment processing | High | Full suite + PCI DSS checklist | Two senior engineers | Security + compliance |
Escalation path: Any automated check failure on high-risk code blocks the merge. Medium-risk failures require senior engineer override with documented justification. Low-risk failures can be addressed in follow-up commits.
Review your framework quarterly. As AI tools evolve and your team learns which checks catch real issues versus noise, adjust the classification criteria and automation rules. Your verification framework is a living system, not a one-time implementation.
