What Happened
Novee's security research team discovered a vulnerability in Cursor, an AI-powered code editor, that allowed full remote code execution on a developer's workstation. The flaw exploited the AI assistant's ability to interpret and execute commands, turning the development tool into an attack vector. An attacker could craft prompts that caused Cursor to execute arbitrary code with the developer's privileges.
This wasn't just theoretical. Novee disclosed the vulnerability to Cursor's team, who patched it. But the incident reveals a critical gap: your security testing infrastructure likely can't detect this class of vulnerability because traditional scanners don't understand how large language models (LLMs) process and act on input.
Timeline
Discovery Phase: Novee's research team identified the vulnerability while testing their AI pentesting agent against LLM-powered applications.
Disclosure: Novee reported the vulnerability to Cursor's development team through responsible disclosure channels.
Remediation: Cursor patched the vulnerability (specific timeline not publicly disclosed).
Public Disclosure: Novee referenced the vulnerability as a demonstration of AI-specific attack vectors that traditional security tools miss.
Which Controls Failed or Were Missing
The Cursor vulnerability exposes three control failures:
Input Validation on AI-Interpreted Commands: Cursor didn't properly sanitize or validate prompts before the LLM processed them. The AI assistant treated certain prompt patterns as executable instructions rather than user requests. This differs from SQL injection or XSS because the attack surface is the model's interpretation layer, not a parsing engine.
Separation of AI Decision-Making from System Execution: The application lacked adequate boundaries between what the AI could recommend and what it could directly execute. Your traditional application firewall won't catch this because the malicious payload doesn't look like code until the LLM interprets it.
AI-Specific Threat Modeling: Cursor's development team likely used standard threat modeling frameworks (STRIDE, PASTA) that don't account for prompt injection, jailbreaking, or model manipulation attacks. These frameworks assume deterministic input processing. LLMs are probabilistic systems with emergent behaviors that traditional models can't predict.
You can't pen test an AI application the same way you test a REST API. Your scanner sends payloads designed to trigger parsing errors or expose unvalidated input. But LLM vulnerabilities often require contextual conversation, multi-turn attacks, or semantic manipulation that looks benign to signature-based detection.
What the Relevant Standards Require
OWASP ASVS v4.0.3, Requirement 5.1.1 mandates input validation using positive validation (allowlisting) and specifies that validation should occur on a trusted system. For AI applications, this means validating prompts before they reach the model and validating model outputs before execution. Cursor failed the second check.
NIST 800-53 Rev 5, Control SI-10 (Information Input Validation) requires organizations to check information inputs for accuracy, completeness, validity, and authenticity. The control explicitly states that validation should prevent "improper commands" from executing. In an AI context, this extends to preventing prompt injection attacks that cause the model to generate improper commands.
ISO 27001:2022, Control 8.24 (Use of Cryptography) doesn't directly apply here, but the broader principle does: protect data in transit and at rest. For AI applications, you need to protect the prompt chain and model outputs as they move through your system. If an attacker can inject malicious instructions into the conversation history, they can manipulate future model behavior.
PCI DSS v4.0.1, Requirement 6.4.3 addresses custom software development and requires security testing before release. The requirement specifies testing for "common software attacks" but doesn't define what "common" means for AI applications. Your QA team running Burp Suite against an LLM-powered checkout flow won't catch prompt injection vulnerabilities because those tools don't simulate conversational attacks.
The standards assume you know what to test for. They don't tell you how to test probabilistic systems that can be manipulated through natural language.
Lessons and Action Items for Your Team
Map Your AI Attack Surface: List every application in your environment that uses an LLM. Include obvious ones (chatbots, code assistants) and hidden ones (ticket routing systems, log analysis tools, automated code review). For each application, document:
- What actions can the AI initiate?
- What system resources can it access?
- How does user input reach the model?
- How do model outputs trigger system actions?
Extend Your Threat Model: Add AI-specific attack patterns to your threat modeling sessions. Start with the OWASP Top 10 for LLM Applications (2023 edition). Key threats include prompt injection (LLM01), insecure output handling (LLM02), and excessive agency (LLM08). Run through each threat against your AI applications just like you'd test for SQL injection.
Implement Dual Validation: Validate input before it reaches the model (traditional input validation) and validate model output before execution (AI-specific validation). For the Cursor vulnerability, this means checking that the AI's suggested commands match expected patterns before executing them. Don't trust the model to self-regulate.
Test with AI-Aware Tools: Your existing DAST and SAST tools can't detect prompt injection or jailbreaking attempts. You need tools that understand conversational context and can simulate multi-turn attacks. Novee's agent is designed for this, but you can start simpler: manually test your AI applications with adversarial prompts. Try to make the model ignore its instructions, leak system prompts, or execute unintended actions.
Separate Privileges: Apply least privilege to AI systems. If your code assistant only needs to read files and suggest edits, don't give it write access or execution permissions. The Cursor vulnerability was severe because the AI could execute code. Limit what your AI can do, even if it's compromised.
Monitor AI Behavior: Log all prompts, model responses, and actions taken by AI systems. Set up alerts for unusual patterns: sudden changes in command frequency, attempts to access restricted resources, or outputs that match known attack signatures. You're looking for the AI equivalent of lateral movement.
The Cursor incident isn't an isolated case. It's a preview of what happens when you deploy AI without adapting your security controls. Your compliance checklist says you need input validation and security testing. Make sure those controls actually work for the systems you're building now.
