When the AI Agent You Built Stops Working

You developed an AI security agent in just three days. It scanned your repositories, flagged secrets, and impressed your VP. Two months later, it's generating 400 false positives per day, and your team has stopped reading its alerts.

This scenario is common when organizations treat AI security agents as quick projects rather than robust production infrastructure.

Understanding the Problem

Organizations often build AI security agents using advanced models to automate vulnerability detection, secret scanning, and compliance checks. These prototypes perform well in controlled settings. However, when deployed to production environments with diverse codebases, accuracy declines. Alert fatigue sets in, and teams revert to manual processes or abandon the agents.

The issue isn't with the AI model itself but with the operational layer: context engineering and harness engineering. The systems lack the infrastructure to maintain accuracy as codebases evolve, dependencies change, and new frameworks appear.

Timeline of Events

Week 1-2: Initial Build
Security teams create prototypes using models like GPT-4 or Claude. Basic functions work: scanning code, identifying patterns, generating alerts. Stakeholders are impressed.

Week 3-8: Production Deployment
Agents connect to live repositories. Initial results are promising. Teams configure alerting and integrate with ticketing systems.

Week 9-16: Degradation
False positive rates increase. Agents misidentify safe patterns as vulnerabilities and miss issues in new frameworks. Context drift occurs as codebases evolve, but the agent's understanding doesn't.

Week 17+: Abandonment or Crisis
Teams either stop using the agent or scramble to rebuild context systems and evaluation frameworks. The "three-day prototype" becomes a six-month operational commitment.

Identifying Control Failures

Context Engineering Failure
Agents lack accurate, current information about the codebase. They fail to understand:

• Which dependencies are in use versus unused
• Internal authentication patterns that resemble hardcoded credentials
• Framework-specific security controls that appear absent
• Business logic that justifies certain data flows

Without this context, models default to generic security patterns that don't match the organization's actual risk profile.

Harness Engineering Absence
Agents lack operational infrastructure:

• No evaluation framework to measure accuracy over time
• No feedback loops for retraining or adjusting based on false positives
• No orchestration layer for handling rate limits, retries, or API failures
• No security controls around the agent's access to sensitive code

Change Management Gap
Teams consider the agent "done" after initial deployment. No process exists to update context as:

• New frameworks are adopted
• Security policies change
• Development patterns evolve
• New team members join with different coding styles

Relevant Standards and Requirements

ISO/IEC 27001:2022 Annex A.8.32 (Change Management)
Requires documented procedures for changes to information processing facilities and systems. An AI security agent is an information processing system. Its context and evaluation criteria must be versioned and updated through change control.

Your agent needs:

• A documented baseline of its knowledge about your codebase
• A change process for adopting new frameworks or patterns
• Version control for context documents and evaluation datasets

NIST CSF v2.0 Detect (DE.CM-1)
"Networks and network services are monitored to find potentially adverse events." An AI agent generating high volumes of false positives fails this control, obscuring adverse events in noise.

You must:

• Measure and track your agent's precision and recall
• Set thresholds for acceptable false positive rates
• Disable or retrain the agent when it crosses those thresholds

SOC 2 Type II CC7.2 (System Monitoring)
Requires monitoring of system components and the operation of controls. Your AI agent is now a control. You need evidence that it's working correctly.

Document:

• Weekly accuracy metrics (true positives, false positives, false negatives)
• Incidents where the agent failed to detect a known vulnerability type
• Context updates and their impact on detection rates

OWASP ASVS v4.0.3 Requirement 14.1.3
Build and deployment pipelines must include security automation that doesn't introduce delays exceeding risk tolerance. An AI agent requiring manual review of 400 false positives per day introduces unacceptable delay.

Your threshold: If manual review time exceeds the time saved by automation, the control has failed.

Lessons and Action Items for Your Team

Treat Context as Code
Your agent's context documents—what it knows about your codebase, frameworks, and policies—must be versioned in git. Update the context when adopting a new framework or changing authentication patterns.

Action: Create a ai-agent-context/ directory in your security repo. Include:

• frameworks.md: List of frameworks in use and their security patterns
• exceptions.md: Known patterns that look suspicious but are approved
• policies.md: Your organization's security policies in plain language

Update this quarterly or when adopting new technology.

Build Evaluation Before Deployment
Create a test dataset of 100 real code samples: 50 with known vulnerabilities, 50 clean. Run your agent against this dataset weekly. Track precision and recall.

Action: If precision drops below 80% (more than 1 in 5 alerts is false), stop deployment and retrain. If recall drops below 90% (missing more than 1 in 10 real issues), the agent is creating risk.

Plan for Operational Overhead
Budget 4-8 hours per week for agent maintenance:

• Review false positives and update context
• Add new test cases to your evaluation dataset
• Monitor API costs and rate limits
• Update the agent when models are deprecated

Action: Assign this to a specific engineer. It's not "everyone's job"—it needs an owner.

Implement Feedback Loops
When a developer marks an alert as a false positive, capture why. Feed this back into context.

Action: Add a "Why is this wrong?" field to your alert dismissal workflow. Review these monthly and update context documents accordingly.

Set Kill Criteria
Define the conditions under which you'll shut down the agent.

Action: Document in your runbook:

• False positive rate exceeds 30%
• Agent misses a critical vulnerability found by other means
• Operational cost (engineer time + API costs) exceeds $X per month
• Agent hasn't been updated in 90 days

The challenge isn't building an AI security agent; it's running one reliably for 18 months. Treat it like production infrastructure—with change management, monitoring, and an operational budget—or don't deploy it at all.