Skip to main content
The Visibility Gap: What Happens When Half Your Team Can't Track AI-Generated CodeStandards
5 min readFor Security Engineers

The Visibility Gap: What Happens When Half Your Team Can't Track AI-Generated Code

The Challenge

Your security team faces a new measurement problem: you can't quantify what you're protecting. According to Checkmarx's Future of AppSec Report, nearly half of organizations don't know what percentage of their application code comes from AI generation tools like GitHub Copilot, ChatGPT, or Amazon CodeWhisperer.

This gap is significant. When your developers use AI assistants to write authentication flows, API endpoints, or database queries, that code ships to production with the same privileges as human-written code. However, your Static Application Security Testing (SAST) tools scan it the same way they scanned code in 2019—looking for patterns they were trained to recognize in human coding habits.

The problem grows with the speed of AI-generated code. It accelerates development cycles, meaning more code reaches production faster, with less time for traditional security review. Your SAST tools flag thousands of potential issues, but they can't distinguish which vulnerabilities matter in your running application versus false positives in unused code paths.

The Environment and Constraints

Security teams in AI-driven development environments face three main constraints:

Visibility Blindness: Your tools don't differentiate between human-written and AI-generated code. When a vulnerability appears in production, you can't trace whether it originated from a developer's implementation or an AI suggestion that seemed correct but introduced a flaw.

Tool Inadequacy: SAST operates on static code analysis—it reads source files without executing them. This works with predictable patterns, but AI models generate code based on statistical likelihood, not security principles. They may produce authentication bypasses that look correct but fail under runtime conditions.

Resource Pressure: Your team already triages hundreds of SAST findings weekly. Adding AI-generated code volume without detection capabilities means either accepting more risk or overloading your security engineers with manual reviews.

The key need: runtime verification, not just static analysis. You need to see what actually happens when AI-generated authentication logic runs against your production environment, when AI-written SQL queries interact with your database, and when AI-created API endpoints handle malformed input.

The Approach Taken

Organizations addressing this gap are implementing Dynamic Application Security Testing (DAST) as a necessary complement to their existing toolchain. The data shows adoption momentum: 47% of organizations plan to have DAST in place by 2025, up from 38% the previous year.

DAST operates differently than SAST. Instead of reading code, it attacks your running application—sending malicious payloads, testing authentication boundaries, and probing for injection vulnerabilities. This approach reveals what AI-generated code actually does under adversarial conditions.

Simon Bennetts, ZAP Software Engineering Expert, emphasizes that DAST's value lies in runtime insight. When your AI assistant generates an input validation function, DAST tests whether that function actually blocks SQL injection attempts, not just whether the code looks like it should.

The implementation pattern emerging across security teams:

Stage 1: Deploy DAST against staging environments running the same AI-generated code that will reach production. This catches runtime vulnerabilities before they ship.

Stage 2: Integrate DAST into CI/CD pipelines as a blocking gate for high-severity findings. If AI-generated authentication code allows privilege escalation during DAST testing, the build fails.

Stage 3: Use AI to enhance DAST efficiency—not replace it. Machine learning models can prioritize which endpoints to test more aggressively, predict likely vulnerability locations, and reduce false positive rates.

The critical insight: AI doesn't replace DAST; it makes DAST essential while simultaneously improving it.

Results and Metrics

The adoption numbers tell part of the story: a 24% year-over-year increase in planned DAST deployment represents the fastest growth in the AppSec tool category. Organizations aren't replacing SAST—they're adding DAST because SAST alone can't verify runtime behavior.

The visibility gap remains the driving concern. When half your organizations can't measure AI-generated code percentage, you can't calculate risk exposure. DAST provides a different measurement: it shows you what vulnerabilities exist in your running application regardless of code origin.

Teams implementing DAST alongside AI-driven development report finding vulnerability classes that SAST missed: authentication bypasses that only trigger under specific session states, API endpoints that leak data when called in unexpected sequences, and race conditions in AI-generated concurrency code.

What They Would Do Differently

Security teams adopting DAST in AI-driven environments identify three areas for improvement:

Earlier Integration: Waiting until AI-generated code reaches staging means finding issues late. Teams wish they'd implemented DAST in development environments where developers could see runtime results immediately after AI assistance.

Better Instrumentation: DAST finds vulnerabilities but doesn't always provide enough context for quick remediation. Teams want deeper integration between DAST findings and the specific AI prompts or suggestions that generated vulnerable code.

Clearer Ownership: When DAST flags an AI-generated vulnerability, who fixes it—the developer who accepted the suggestion, the security team, or the platform team managing AI tools? Organizations wish they'd established this accountability before deployment.

Takeaways for Your Team

If you're running security in an environment where developers use AI coding assistants:

Measure Your Visibility Gap First: Survey your development teams about AI tool usage. If you can't quantify how much AI-generated code exists in your applications, you can't scope your security tooling needs.

Plan DAST Deployment Now: The 47% adoption target for 2025 means your peers are already implementing runtime testing. Evaluate DAST tools against your specific application architecture—API-heavy applications need different DAST capabilities than monolithic web applications.

Integrate, Don't Replace: DAST complements SAST; it doesn't obsolete it. Your SAST tools still catch many issues before code runs. DAST verifies what actually happens when that code executes.

Use AI to Improve DAST: Machine learning can optimize test case generation, reduce scan times, and prioritize findings. This isn't about replacing security engineers—it's about focusing their expertise on issues that matter.

Establish Clear Remediation Workflows: Before you deploy DAST, define who responds to findings in AI-generated code. Document whether developers need to modify prompts, adjust AI suggestions, or write manual fixes.

The fundamental shift: AI-generated code moves faster than human-written code, which means your security verification must operate at runtime, not just at rest. DAST provides that runtime verification. The question isn't whether you need it—the adoption data shows that answer. The question is whether you implement it before or after you discover vulnerabilities in production.

Dynamic Application Security Testing (DAST)

Topics:Standards

You Might Also Like