In early 2024, researchers monitoring cybercrime forums documented a seven-month conversation among threat actors exploring AI tools. The analysis covered more than 160 forum threads where criminals discussed using large language models to automate phishing, generate polymorphic malware, and scale reconnaissance operations.
This wasn't a breach. No company lost data. But the pattern reveals a control gap that most security teams haven't addressed: you're building defenses against yesterday's attacks while adversaries prototype tomorrow's.
What Happened
Between mid-2023 and early 2024, cybercriminals on underground forums ran practical tests with publicly available AI models. The conversations split into two camps. Some threat actors shared scripts that used GPT-based models to generate convincing spear-phishing emails at scale, bypassing traditional keyword filters. Others expressed skepticism, noting that AI-generated content often included detectable patterns and that existing tools still worked fine.
The forums showed adversaries weren't just theorizing. They documented actual experiments: using AI to rewrite malware signatures, automate OSINT collection against target organizations, and generate context-aware social engineering hooks based on scraped LinkedIn profiles.
Timeline
Mid-2023: Initial forum threads appear discussing ChatGPT's potential for phishing campaigns.
September-October 2023: Threat actors share early scripts combining web scraping with LLM-generated email templates.
November 2023-January 2024: More sophisticated discussions emerge around using AI for code obfuscation and polymorphic malware development.
February 2024: Research analysis published covering 160+ conversations across multiple forums.
Which Controls Failed or Were Missing
Your threat modeling probably doesn't account for AI-augmented attacks. Here's what that gap looks like in practice:
Email security controls assume static patterns. Your spam filters flag known phishing templates and suspicious keywords. They don't catch AI-generated emails that adapt tone, vocabulary, and context to match your organization's communication style. The threat actors documented success rates improving when they used scraped internal communications to fine-tune their prompts.
Signature-based malware detection breaks down. If an adversary can regenerate their payload with different variable names, function orders, and code structures every few minutes, your endpoint detection needs behavioral analysis, not hash matching. The forum conversations showed criminals testing exactly this approach.
OSINT collection happened outside your visibility. When someone uses an AI model to aggregate and synthesize your public job postings, GitHub repositories, and conference presentations into a target profile, you won't see scanning traffic or failed authentication attempts. The reconnaissance happens in their environment, not yours.
Security awareness training targets old threats. You're training employees to spot Nigerian prince emails and obvious typos. The forum posts described AI-generated phishing that referenced real projects, used accurate internal terminology, and matched the writing style of actual executives.
What the Standards Require
The gap isn't that standards ignore emerging threats. It's that you need to map AI-augmented attacks to controls you already committed to implementing.
NIST CSF v2.0 requires continuous threat intelligence monitoring under the Identify function (ID.RA-03: "Threats, both internal and external, are identified and documented"). If you're not tracking how adversaries discuss and test AI tools in criminal forums, you're missing threat intelligence that directly affects your risk posture.
ISO 27001 Annex A.5.7 mandates threat intelligence processes. The standard expects you to understand "information about threats that could exploit vulnerabilities." Cybercriminals openly testing AI attack methods counts as threat intelligence you should be collecting and acting on.
PCI DSS v4.0.1 Requirement 12.6.3 requires security awareness training that addresses current threats. If your training materials don't cover AI-generated phishing and social engineering, you're not meeting the "current" threshold. The requirement specifically calls out training content that reflects "the latest threats and vulnerabilities."
NIST 800-53 Rev 5 Control SI-4 (System Monitoring) and AT-2 (Literacy Training and Awareness) both require you to adjust defenses and training based on emerging threats. The control enhancements explicitly mention monitoring for "unusual or unauthorized activities."
Lessons and Action Items for Your Team
Update your threat model this quarter. Add a section specifically addressing AI-augmented attacks. Document which controls would detect or prevent: polymorphic malware generated via LLMs, AI-enhanced spear phishing, and automated OSINT collection. If you can't point to specific detective or preventive controls for each scenario, you've found your gap.
Shift email security from signatures to behavior. Your spam filter needs to analyze email behavior patterns, not just content. Look for: unusual sending patterns from legitimate accounts, emails that reference non-public information but come from external sources, and messages that perfectly match your communication style but originate outside your domain. These patterns matter more than keyword matches when adversaries use AI to generate content.
Implement behavioral endpoint detection now. If you're still primarily using signature-based antivirus, you're vulnerable to AI-generated polymorphic malware. Move to EDR that monitors process behavior, memory access patterns, and lateral movement attempts. The control needs to catch malicious behavior regardless of how the code is structured.
Run tabletop exercises against AI scenarios. Your next security drill should include: an AI-generated phishing campaign that bypasses your filters, malware that rewrites itself every deployment, and a social engineering attack that references accurate internal details. Document which controls would catch each stage and where you'd be blind.
Revise security awareness training by next cycle. Add modules showing employees what AI-generated phishing actually looks like. Include examples that use correct terminology, reference real projects, and match executive communication styles. The training needs to shift from "spot the typo" to "verify through a second channel."
Monitor criminal forums for your industry. You don't need to do this in-house. Threat intelligence services already monitor these conversations. Subscribe to one that covers your sector and set up alerts for discussions about AI tools, your technology stack, or your industry's common vulnerabilities.
The forum conversations showed cybercriminals testing AI tools the same way your team tests new security products. They're documenting what works, sharing scripts, and iterating on techniques. Your controls need to evolve at the same pace.
