The Growing Security Debt in AI
Organizations rushing to deploy AI features are creating a security backlog they cannot manage. According to Cobalt's AI and Pentesting Pulse Report 2026, AI and LLM applications exhibit high-risk vulnerabilities at 2.7 times the rate of traditional systems. Alarmingly, only 38.4% of serious AI findings are resolved, leaving two-thirds of critical vulnerabilities in AI systems unaddressed.
This issue spans industries. Teams implement chatbots, recommendation engines, and natural language interfaces without adequate security controls. Shadow AI—unauthorized AI tools deployed by individual teams—accounts for 44% of confirmed AI security incidents. Security leaders and practitioners report a 42-point gap in their assessment of remediation success, with leaders believing the work is done, while practitioners know it isn't.
A Timeline of AI Vulnerability
Q1 2024: Organizations rapidly deploy AI features to stay competitive, prioritizing speed over security.
Q2-Q3 2024: Security teams identify AI-specific vulnerabilities like prompt injection attacks and model extraction attempts, which queue behind existing backlogs.
Q4 2024: Shadow AI emerges as teams bypass procurement to access commercial LLM APIs directly, reducing security visibility.
Q1 2025: Incident response teams handle breaches involving AI components, with common attack vectors including exposed API keys and unprotected model endpoints.
2026: The resolution rate for high-risk AI findings stabilizes at 38.4%, less than half the rate for traditional application vulnerabilities.
Missing or Failed Controls
Asset Inventory and Discovery: Effective security requires visibility. Shadow AI thrives due to incomplete inventories of AI components. When 44% of AI incidents involve unauthorized tools, asset management has failed.
Input Validation and Sanitization: AI applications often accept natural language input, which existing validation frameworks cannot handle. Prompt injection attacks succeed because teams apply input filters designed for structured data.
Access Controls for Model Endpoints: Many organizations use the same authentication for LLM APIs as for CRUD operations. Model endpoints need different controls, such as rate limiting and prompt logging.
Vulnerability Management Prioritization: The 61.6% of unresolved AI findings indicate flawed triage processes. Traditional CVSS scoring misprioritizes AI-specific risks.
Security Awareness and Training: Developers often lack training in AI-specific attack vectors. They may understand SQL injection but not model extraction, validating form inputs but not prompt boundaries.
Compliance Standards and Requirements
PCI DSS v4.0.1 Requirement 6.3.2 mandates secure development of custom software based on industry standards. For AI components processing payment data, this includes:
- Threat modeling for prompt injection and model manipulation
- Security testing for LLM attack patterns
- Code review processes for AI-specific vulnerabilities
OWASP Top 10 for LLMs outlines ten critical risks to address:
- LLM01: Prompt Injection
- LLM02: Insecure Output Handling
- LLM03: Training Data Poisoning
- LLM06: Sensitive Information Disclosure
ISO/IEC 27001:2022 Annex A.8.1 requires asset inventory and acceptable use policies. Shadow AI violates both. You need:
- A complete inventory of AI tools and integrations
- Clear policies for approved AI services
- Technical controls to detect unauthorized AI API calls
NIST Cybersecurity Framework v2.0 calls for continuous asset discovery and vulnerability identification. The 44% shadow AI incident rate indicates failure in these areas.
SOC 2 Type II CC6.1 requires monitoring and logging of system access. For AI systems, this means:
- Logging all prompts and model responses
- Monitoring for unusual query patterns
- Alerting on potential data exfiltration through prompt engineering
Action Items for Your Team
1. Build an AI-Specific Asset Inventory
Conduct network traffic analysis to identify LLM API calls. Search your codebase for imports of OpenAI, Anthropic, Google AI, and similar SDKs. Compare findings against your approved vendor list to identify shadow AI.
2. Separate AI Vulnerability Triage
Create a distinct queue for AI/LLM findings. Train your team on AI-specific risk scoring. Traditional CVSS scoring may undervalue risks like prompt injection that expose training data.
3. Implement Prompt Logging and Monitoring
Ensure comprehensive logging for every production LLM endpoint:
- Full prompt text (sanitized for PII)
- Model responses
- User context and session data
- Response time and token count
Set alerts for repeated similar prompts, unusually long prompts, and high-volume queries from single users.
4. Close the Leader-Practitioner Perception Gap
Track metrics like time-to-remediation by vulnerability type and reopen rates for AI-specific findings. Share these metrics to align leaders and practitioners on progress and challenges.
5. Require AI Security Training
Before deploying AI features, ensure developers complete training on:
- OWASP Top 10 for LLMs
- Prompt injection attack patterns
- Secure model deployment practices
- Data handling requirements
6. Establish an AI Security Review Gate
Add a mandatory security review for features using external LLM APIs, locally hosted models, vector databases, or retrieval-augmented generation systems. Conduct this review before code review to address architectural decisions early.
The 61.6% unresolved AI vulnerability rate is a prioritization problem. Address it by treating AI components as high-risk systems and allocating resources accordingly.
