Your team ships code 55% faster with AI assistance. But when that code breaks in production at 2 AM, can they fix it?
This isn't a hypothetical problem. According to Octopus Deploy research, 73% of organizations have reduced junior hiring over the past two years—right as AI coding tools became standard. We're creating a generation of developers who can generate code but struggle to understand what it does or why it fails.
For security engineers, this matters more than a productivity metric. When developers can't trace execution flow or read stack traces without AI assistance, they can't implement security controls correctly. They can't spot the difference between a safe input validation and a bypassable one. And they definitely can't respond to security incidents that require understanding code behavior under attack conditions.
What This Checklist Covers
This checklist helps you assess whether your development team has the debugging and code comprehension skills needed to maintain security standards in an AI-assisted workflow. Use it quarterly to identify skill gaps before they become security incidents.
Prerequisites
Before using this checklist, you need:
- Access to recent code reviews - Pull requests from the last 30 days where AI tools were used
- Incident response records - Postmortems from the last quarter, especially security-related bugs
- Training records - What debugging training your team has completed
- Tool logs - Usage patterns for AI coding assistants like Claude Code or GitHub Copilot
Checklist Items
1. Code Understanding Without AI
Check: Select 3 pull requests where AI was heavily used. Ask the author to explain the code's execution flow without referencing the AI tool or running the code.
Good looks like: Developer walks through the logic, explains edge cases, identifies potential failure modes. They can answer "what happens if this API returns a 429?" or "how does this handle null values?"
Not good: Developer says "the AI wrote this part" or needs to ask the AI to explain their own code.
2. Stack Trace Interpretation
Check: Present developers with a stack trace from your application (sanitize sensitive data). Ask them to identify the root cause without using AI tools.
Good looks like: Developer traces the error back through the call stack, identifies the failing line, proposes 2-3 potential causes based on the context.
Not good: Developer immediately pastes the stack trace into an AI tool or can't identify which file contains the problem.
3. Security Control Implementation
Check: Review implementations of authentication, input validation, or authorization logic added in the last quarter. For each, verify the developer can explain why the control works and what attacks it prevents.
Reference: OWASP ASVS v4.0.3 Section 4.1 (Access Control) and Section 5.1 (Input Validation)
Good looks like: Developer explains the threat model, walks through bypass attempts that would fail, identifies related controls that work together.
Not good: Developer says "this is what the AI suggested" or can't explain what happens if an attacker sends unexpected input.
4. Debugging Tool Proficiency
Check: Observe developers debugging a production issue. Do they use debuggers, logging, profilers, or network inspection tools?
Good looks like: Developer sets breakpoints, inspects variable state, traces network requests, reads logs systematically. They form hypotheses and test them.
Not good: Developer's first action is describing the problem to an AI tool. They don't know how to attach a debugger or read application logs.
5. Code Review Quality
Check: Examine code reviews where junior developers reviewed AI-generated code. Look for comments about logic errors, security issues, or edge cases.
Reference: PCI DSS v4.0.1 Requirement 6.3.2 (code reviews must identify coding vulnerabilities)
Good looks like: Reviewers catch issues like SQL injection risks, missing error handling, race conditions, or authentication bypasses. Comments show they read and understood the code.
Not good: Reviews consist of "LGTM" or only surface-level style comments. Security issues pass through unnoticed.
6. Incident Response Capability
Check: During your last security incident or critical bug, how long did it take to identify the root cause? Who could actually trace through the code?
Good looks like: Team isolated the vulnerable code within hours. Multiple developers could explain the bug and propose fixes. Postmortem includes clear technical analysis.
Not good: Only senior developers could debug the issue. Team needed to ask AI tools to explain the codebase. Resolution took days because no one understood the affected code.
7. Training Investment
Check: What debugging training has your team completed in the last 12 months? Does onboarding include debugging practice?
Reference: ISO 27001 Annex A.6.3 (awareness, education and training)
Good looks like: Quarterly debugging workshops. New hires spend their first week debugging intentionally broken code. Team has access to debugging courses and uses them.
Not good: Last training was generic "how to use AI tools." No structured debugging practice. Onboarding focuses on shipping features fast.
Common Mistakes
Treating AI output as trusted code. Just because AI generates syntactically correct code doesn't mean it's secure or correct. Your team needs to review AI suggestions with the same scrutiny as code from an untrusted contractor.
Skipping junior hiring entirely. The 73% reduction in junior hiring creates a pipeline problem. In five years, you won't have mid-level developers who grew up debugging complex systems. Consider hiring juniors specifically to build debugging skills before AI assistance.
Measuring only velocity. Yes, tasks complete 55% faster with AI. But if those tasks create technical debt or security vulnerabilities that take months to fix, you haven't gained speed—you've borrowed it from your future self.
No AI-free practice time. If developers only debug with AI assistance, they never build the mental models needed to solve novel problems. Dedicate time for debugging without AI tools.
Next Steps
If you found gaps in items 1-4, your immediate security risk is high. Developers who can't debug their own code can't implement security controls reliably.
This week:
- Run the stack trace test with your team
- Review your three most recent security fixes—who could actually explain them?
This month:
- Add debugging scenarios to your onboarding
- Require code reviews to include "explain this logic" comments
- Schedule debugging workshops focused on your actual codebase
This quarter:
- Audit security control implementations from the last 90 days
- Update your hiring criteria to assess debugging skills explicitly
- Build a library of debugging exercises using real bugs from your system
The productivity gains from AI are real. But if your team can't debug the code they ship, you're trading short-term velocity for long-term technical debt—and security risk. Make sure your developers can still do the job when the AI isn't available.
