What Happened
A development team using LLM-assisted code generation released multiple pull requests containing hardcoded API keys, database credentials, and overly permissive access controls. The code worked perfectly in testing, passed automated linting, and deployed to production without triggering any alerts.
The issue: AI-generated code can expose API keys, tokens, credentials, or permissive defaults. When you ask an LLM to "build a payment processing module" or "connect to our database," it generates functional code based on patterns from public repositories. These patterns often include placeholder credentials, overly broad permissions, and authentication bypasses.
Your code review missed these issues because reviewers focused on logic and functionality, not scanning for secrets. Your secret scanning tools failed because the credentials were newly generated and didn't match known patterns.
Timeline
Week 1, Monday: Product team requests a new customer dashboard with real-time analytics. Developer uses LLM to scaffold the API layer, database connections, and authentication flow.
Week 1, Wednesday: Code review focuses on business logic and UI/UX. Three engineers approve the PR. Nobody questions the hardcoded JWT secret in the config file because it "looks like a placeholder."
Week 1, Friday: CI/CD pipeline runs SAST tools configured to check for known vulnerability patterns. The tools flag SQL injection risks, which the developer fixes, but miss the embedded credentials because they're formatted as environment variable assignments.
Week 2, Monday: Code deploys to production. The authentication service uses the hardcoded JWT secret. The database connection string includes an admin password. The S3 bucket policy defaults to public-read.
Week 2, Thursday: An external researcher discovers the exposed S3 bucket through automated scanning. Your security team learns about it from a responsible disclosure email, not from internal monitoring.
Which Controls Failed or Were Missing
Secrets Management: No pre-commit hooks scanning for credential patterns. No requirement that all secrets must be retrieved from a vault or secrets manager at runtime. Developers could commit working code with inline credentials unchecked.
Code Review Process: Reviewers evaluated whether the code solved the business problem, not whether it met security requirements. No checklist ensured authentication mechanisms, access controls, and credential handling followed secure patterns.
SAST Tool Configuration: Your static analysis tools checked for OWASP Top 10 vulnerabilities but weren't tuned to catch AI-specific risks like permissive defaults, placeholder credentials, or authentication logic that violates the principle of least privilege.
Runtime Validation: No automated testing verified credentials came from approved sources. No policy enforcement prevented deployment if secrets were detected in the artifact.
Dependency Verification: The LLM pulled in several third-party libraries to handle authentication. Nobody validated those dependencies against an approved list or checked them for known vulnerabilities before they shipped to production.
What the Relevant Standard Requires
PCI DSS v4.0.1 Requirement 6.3.2 mandates that custom software be developed securely based on industry standards. This includes secure authentication, secure communication, and secure coding techniques. Hardcoded credentials and permissive defaults violate this requirement.
OWASP ASVS v4.0.3 Section 2.7 requires that secrets, API keys, and passwords are not hardcoded in application code. They must be stored in a secure credential store and retrieved at runtime.
NIST 800-53 Rev 5 Control IA-5 specifies that authenticators must have sufficient strength for their intended use and must be protected against unauthorized disclosure. A hardcoded JWT secret fails both tests.
SOC 2 Type II Common Criteria CC6.1 requires that logical and physical access controls restrict access to information assets. Public S3 buckets and overly permissive database credentials violate this control.
These standards focus on the security properties of what you deploy, regardless of whether a human or an LLM wrote the code.
Lessons and Action Items for Your Team
Implement Pre-Commit Secret Scanning: Install tools like Talisman, git-secrets, or Gitleaks as pre-commit hooks. Configure them to block commits containing patterns that match API keys, private keys, passwords, or tokens. Update the pattern library monthly as new services and credential formats emerge.
Require Secrets Manager Integration: Mandate that all credentials must be retrieved from HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or your approved secrets management platform. No exceptions. If a developer claims they need to hardcode something "temporarily," the answer is no.
Build an AI-Aware Code Review Checklist: Add specific items to your review template: "Does this code retrieve credentials from the secrets manager?" "Are all access controls set to least privilege?" "If this code handles authentication, does it follow our approved patterns?" Train reviewers to question anything that looks like a credential, even if it's commented as a placeholder.
Tune Your SAST Tools for AI-Generated Patterns: Work with your AppSec team to create custom rules that catch permissive defaults, placeholder-looking credentials, and authentication logic that deviates from your approved libraries. Don't assume tools trained on human-written code will catch AI-specific risks.
Gate Deployments on Security Validation: Add a pipeline stage that fails the build if secrets are detected in the artifact, if dependencies aren't on your approved list, or if runtime testing shows the application attempting to use hardcoded credentials. Make it impossible to deploy insecure code, even if it passes functional tests.
Test Your Authentication at Runtime: Write integration tests that verify credentials are loaded from the expected source. If your app can start up without connecting to the secrets manager, your test should fail.
You can't slow down AI-assisted development, and you shouldn't try. What you can do is build security checks that run at the same speed as code generation. When your developer asks an LLM to build a feature, your security tooling should validate that feature before it reaches production. That's not a future state. That's the requirement right now.
