Skip to main content
When the Vendor Selling the Lock Also Sells the LockpickIncident
4 min readFor CISOs

When the Vendor Selling the Lock Also Sells the Lockpick

What Happened

OpenAI president Greg Brockman published a blog post urging CISOs to adopt agentic AI systems, specifically recommending OpenAI's own products. His pitch is that company systems are hiding significant flaws that attackers will exploit, and autonomous AI agents can find them first. Analysts immediately flagged what the post didn't say: nothing about liability, rollback mechanisms, or what happens when these agents make mistakes in production environments.

This isn't a traditional security incident. It's a vendor pitch disguised as strategic guidance, and it deserves the same scrutiny you'd give any incident that could compromise your security posture.

Timeline of Red Flags

The sequence matters here:

  1. Initial pitch: Brockman frames agentic AI as the solution to hidden vulnerabilities.
  2. Product placement: Specific recommendations for OpenAI's commercial offerings.
  3. Missing context: No discussion of failure modes, liability, or safety controls.
  4. Analyst pushback: Security professionals note the self-serving nature and absence of accountability measures.

The issue isn't the sales pitch itself. Every vendor pitches. The problem is how it positions autonomous agents as a security necessity while skipping the fundamental questions you'd ask about any tool that makes decisions in your environment.

Which Controls Failed or Were Missing

Let's map this to what should have been addressed before recommending autonomous agents for security operations:

Change Management: No discussion of how to test, stage, or roll back agent actions. If you're deploying something that autonomously modifies your environment, you need change control.

Access Control: Agentic systems need credentials to do their work. What scope? What duration? What happens if an agent is compromised or behaves unexpectedly?

Audit Logging: How do you trace what the agent did, why it made that decision, and who approved the underlying logic? Your SOC 2 Type II auditor will ask this. So will your incident response team when something breaks.

Incident Response: What's your playbook when an agent causes an outage? When it exposes data? When it conflicts with another automated system? These aren't hypothetical scenarios.

Vendor Risk Management: The pitch assumes you'll trust a third party to run autonomous operations in your environment. Where's the liability language? What happens when the agent's actions violate a regulation or cause financial loss?

What the Standards Require

Here's what your compliance framework already says about this, even if it doesn't mention AI specifically:

NIST CSF v2.0 (Govern function): GV.RR-03 requires you to understand legal, regulatory, and contractual requirements. If you're deploying agents that make security decisions, you need clear documentation of liability when those decisions go wrong.

ISO/IEC 27001:2022 (Control 5.19 - Information security in supplier relationships): You must assess security risks in supplier relationships. An agent running in your environment with decision-making authority is absolutely a supplier relationship. What's your assessment process?

PCI DSS v4.0.1 (Requirement 6.3.2): Changes to system components must follow change control procedures. If an agent is making changes, those changes need the same controls as human-initiated changes: testing, approval, rollback capability.

SOC 2 Type II (Common Criteria CC7.2): You need monitoring to identify deviations from expected operations. How do you monitor an autonomous agent's decisions? What constitutes "expected" when the system is making novel decisions based on its training?

None of these standards prohibit AI. They require you to apply the same rigor to automated decisions that you apply to human ones. That means understanding failure modes, maintaining audit trails, and knowing who's responsible when things break.

Lessons and Action Items for Your Team

If you're evaluating agentic AI for security operations, here's what to do before you sign anything:

Document decision authority: Write down exactly what the agent can and cannot do. Can it modify firewall rules? Disable accounts? Quarantine systems? Get this in writing and map it to your change control policy.

Establish rollback procedures: Before you deploy, know how to undo what the agent does. If it can make changes, you need a documented process to reverse them. Test this process.

Define liability boundaries: Get explicit contract language about what happens when the agent causes an incident. Who pays for the breach notification? The forensics? The regulatory fines? If the vendor won't commit to this in writing, that tells you something.

Build audit trails: Every agent action needs to be logged with context: what it did, why it decided to do it, what data it used to make that decision. Your log retention policy applies here too.

Start with read-only: Deploy agents in observation mode first. Let them analyze and recommend, but don't give them write access until you've validated their decision-making over weeks or months.

Review your insurance: Talk to your cyber insurance carrier before deploying autonomous agents. Some policies have specific language about AI-driven decisions. Find out now, not after an incident.

Challenge vendor claims: When a vendor says their product will find hidden flaws, ask for specifics. What's the false positive rate? How does it handle edge cases? What happens when it's wrong? If they can't answer these questions with data, walk away.

The promise of AI-driven security is real. So are the risks of deploying systems that make autonomous decisions without adequate controls. Treat vendor pitches like you treat any other input to your security program: verify the claims, document the risks, and make sure you can explain your decision to an auditor.

Your job isn't to adopt the latest technology. It's to protect your organization while meeting your compliance obligations. Sometimes those goals align. Sometimes they don't. Make sure you know which situation you're in before you deploy.

Topics:Incident

You Might Also Like