Skip to main content
Promotional banner for a CUI buildout guide
AI Agents Hunted LiteLLM and Dify for API KeysIncident
5 min readFor Security Engineers

AI Agents Hunted LiteLLM and Dify for API Keys

Amazon's threat intelligence team recently uncovered a campaign where attackers used AI agents to target internet-exposed AI applications, including LiteLLM and Dify, to steal API keys. While details on victim impact are limited, this analysis focuses on the established facts and maps them to controls that could have prevented the attack.

What Happened

Attackers utilized ARTEX, an open-source AI tool, to automate penetration testing of AI applications accessible from the internet. Their goal was to steal API keys that these applications use to connect with upstream model providers and other services.

Amazon detected this activity through its decoy servers, part of a network called MadPot, which captured records of the attack. This evidence comes from honeypot interaction, not from a victim's incident response, providing a rare view into attacker methods without needing a breach disclosure.

Two key points stand out. First, the tooling is open source, making it easily accessible. Second, the targets are software that inherently aggregates secrets. An LLM gateway or workflow builder acts as a vault of provider keys with a web interface.

Timeline

  • October 3: MadPot decoy servers begin capturing records of the campaign.
  • Late last week: Capture window ends. The campaign was ongoing throughout this period.

The timeline indicates that this wasn't a single scan. Activity persisted over an extended period, suggesting that any exposed instance was likely probed multiple times.

Which Controls Failed or Were Missing

Amazon's findings describe attacker behavior rather than specific victim misconfigurations. Consider the following as control gaps this attack pattern exploits.

Internet exposure of management and gateway interfaces. The campaign targeted internet-exposed apps. If an LLM gateway or workflow builder doesn't need public access, exposure is the first failure.

Weak or absent authentication on AI app endpoints. Automated tools quickly find unauthenticated or default-credentialed admin surfaces. Anything that returns configuration or key material without strong authentication is a direct path to theft.

Long-lived, over-scoped API keys stored where the app can read them. Keys held in environment variables, config files, or an app database turn any app compromise into a credential compromise.

No detection of automated probing. An AI agent iterating through an app behaves differently from a human. If your logging can't distinguish that pattern, you won't see it until the keys are used elsewhere.

What the Relevant Standards Require

None of these frameworks mention AI agents specifically, but the requirements apply directly.

OWASP ASVS v4.0.3. Chapter 2 covers authentication, and Chapter 4 covers access control. Requirement 4.1.1 requires that access control is enforced on a trusted service layer, and 4.2.1 requires that sensitive data and APIs are protected against direct object attacks. Chapter 6 (stored cryptography) and the secrets handling guidance in Chapter 14 address how keys are stored and configured.

OWASP Top 10 (2021). A01 Broken Access Control and A07 Identification and Authentication Failures describe the exact weaknesses automated agents probe for. A05 Security Misconfiguration covers exposed admin interfaces and default settings.

ISO/IEC 27001:2022. Annex A control 8.5 (secure authentication) and 8.3 (information access restriction) apply to who can reach these apps and what they can retrieve. Control 8.24 (use of cryptography) covers key management practices. Control 8.16 (monitoring activities) covers detecting anomalous behavior.

NIST 800-53 Rev 5. AC-3 (access enforcement), IA-5 (authenticator management), SC-12 (cryptographic key establishment and management), and SI-4 (system monitoring) map to the same gaps. IA-5 is the one teams skip for API keys, because it applies to application credentials, not only to user passwords.

SOC 2 Type II. The Common Criteria around logical access (CC6.1 through CC6.3) and system monitoring (CC7.2) are the ones an auditor will test. Because Type II covers operating effectiveness over a period, an exposed gateway that sat open for weeks is a control failure that persists across the whole window.

NIST CSF v2.0. The Protect function (PR.AA for identity and access, PR.PS for platform security) and the Detect function (DE.CM for continuous monitoring) cover the same ground at a program level.

Lessons and Action Items for Your Team

1. Inventory your AI apps like any other internet-facing asset. LLM gateways and workflow tools are often deployed quickly by engineering teams and may skip the usual review. Identify them before an attacker's scanner does. Check your external attack surface for LiteLLM, Dify, and similar tools.

2. Take management interfaces off the public internet. Use a VPN, a zero-trust proxy, or private networking. If a gateway must be reachable, expose only the inference endpoint, not the admin and configuration surface.

3. Require strong authentication on every endpoint. No default credentials, no unauthenticated config routes. Verify this by testing, not by reading documentation. Run the checks from ASVS Chapter 2 and 4 against your deployed instance.

4. Move provider keys out of the app's reach. Store them in a secrets manager and inject them at runtime where your platform allows. Scope each key to the minimum models, projects, and spend limits it needs. A stolen key with a hard spending cap and a narrow scope is an annoyance; an unrestricted one is an incident.

5. Rotate on a schedule and on suspicion. Treat any key held by an exposed app as potentially compromised. Build rotation into your runbook so it takes minutes, not days. This is NIST 800-53 SC-12 in practice.

6. Monitor key usage at the provider, not just the app. Alert on usage from unfamiliar IP ranges, unusual model calls, or sudden spend spikes. Stolen keys get used somewhere other than your app, and the provider's logs are where you'll see it first.

7. Detect automated behavior. Rate-limit and log aggressively on AI app endpoints. Rapid, systematic request patterns are a signal. Feed those logs into your SIEM and write a detection for it.

8. Assume the attacker is faster than you. Automated agents compress the time between discovery and exploitation. Your patch cadence and exposure review need to match that pace, which means shortening the window rather than hoping nobody scans you.

The defensive side of this isn't magic either. The same automation that finds your weak spots can be run by you first. Point your own tooling at your AI apps on a schedule, before someone else does.

Topics:Incident
Promotional banner for the Penetration Report Template Kit

You Might Also Like