What Happened
Google has announced it will no longer accept AI-generated submissions to its Open Source Software Vulnerability Reward Program (VRP). This decision follows a trend of low-quality reports that created more work than value for security teams.
In parallel, Google has joined Anthropic, AWS, Microsoft, and OpenAI in committing $12.5 million to enhance open-source security through the Open Source Security Foundation (OSSF). Part of this funding will support AI tools to help maintainers manage the influx of bug reports, including those generated by AI.
Timeline
Understanding the timeline highlights how quickly AI-generated report issues escalated:
Before the ban: Google's VRP accepted submissions from any source, including AI tools that scanned codebases and generated vulnerability reports.
Decision point: After reviewing submission patterns, Google's security team found that AI-generated reports were consuming excessive triage resources compared to their accuracy and usefulness.
Current state: The VRP now explicitly excludes AI-generated submissions. Meanwhile, the $12.5 million funding initiative progresses, with AI tools being developed to aid maintainers in handling these reports.
Which Controls Failed or Were Missing
This situation isn't a traditional breach but a process failure revealing gaps in handling automated submissions:
No quality gate for automated submissions: Google's VRP lacked criteria to filter low-quality reports before reaching human reviewers. Every submission entered the same triage queue, regardless of source or confidence level.
Missing validation requirements: The program didn't require submitters to validate findings or understand the code context. AI tools can flag potential issues but can't verify exploitability or assess business impact.
Insufficient resource allocation for triage: The volume of AI-generated reports overwhelmed the team's review capacity. When you're triaging hundreds of potential false positives, you're not focusing on real vulnerabilities.
No feedback loop to submitters: There wasn't a mechanism to educate submitters on quality reporting. AI tool users continued submitting low-quality findings without feedback on their approach.
What the Relevant Standard Requires
Mapping this to actual requirements your team needs to meet:
PCI DSS v4.0.1 Requirement 6.3.2 mandates maintaining an inventory of bespoke and custom software and third-party components. AI tools generating reports about unused or unverifiable dependencies create noise in this process.
ISO 27001 Control 5.23 requires evaluating security risks when using external services. If you're using AI tools for security reports, you need documented criteria for reliable reports.
NIST CSF v2.0 function Identify (ID.RA-01) requires identifying and documenting asset vulnerabilities. AI-generated reports that aren't validated don't meet the "documented" threshold because they can't be traced to verified weaknesses.
SOC 2 Type II Common Criteria 6.6 addresses logical and physical access controls for monitoring. Accepting unvalidated automated submissions introduces a control gap in identifying what needs monitoring.
Lessons and Action Items for Your Team
Here's what you can implement this quarter:
Set submission standards: If you run a bug bounty or VRP, define valid report criteria. Require proof of concept, reproduction steps, or evidence that the submitter understands the context. Don't accept "tool says it's vulnerable" as sufficient evidence.
Build a quality rubric for automated findings: Create specific criteria for AI-generated reports. Does it include the vulnerable code path? Can you reproduce it in your environment? Does it explain why it matters to your threat model? If not, it goes back.
Separate automated and human-validated submissions: Create different triage queues. AI-generated reports should go through additional filtering before reaching your security engineers. This could be automated validation, required peer review, or a preliminary assessment by the submitter's team.
Use AI on the defense, not just offense: The $12.5 million funding initiative shows where AI helps: giving maintainers tools to handle report volume. Consider AI for triaging incoming reports, identifying duplicates, or suggesting remediation approaches. Don't use it to generate more reports.
Track time spent on false positives: Measure how much time your team spends on invalid reports. If that number is climbing, you've got a process problem. Set a threshold (e.g., no more than 20% of triage time on false positives) and redesign your intake when you hit it.
Require validation from submitters: Before accepting any report, automated or not, require the submitter to answer: Have you verified this in a test environment? What's the actual impact? What would exploitation look like? These questions force validation before the report reaches you.
Document your AI tool usage: If you're using AI to assist with security work, document what tools you're using, what they're analyzing, and how you validate their output. This matters for ISO 27001 audits and SOC 2 examinations.
AI can generate noise when used to create work for humans. It adds value when it helps humans process work more effectively. Google's decision shows they've learned which side of that line their VRP fell on. Make sure your team knows where yours sits.
