What Happened
Internet-wide scanning has revealed 175,000 Ollama servers with publicly accessible APIs. Ollama, a tool for running large language models locally or on cloud infrastructure, exposes a REST-style API by default. This API allows any client to submit prompts, retrieve responses, and manage models without authentication.
Attackers have exploited these exposed servers in two main ways: accessing the inference API to run arbitrary prompts, potentially extracting sensitive data, and abusing compute resources for cryptomining or other resource-intensive tasks.
The exposure results from a design choice. Ollama's API is bound to 0.0.0.0:11434, listening on all network interfaces. Teams deploying Ollama to cloud instances or containers often overlook this default, assuming the service remains local. Once the server reaches production without network restrictions, it's discoverable via port scans.
Timeline
This isn't a single incident but an ongoing exposure pattern affecting thousands of deployments. The issue exists from the moment you start an Ollama server without network controls until you implement proper restrictions.
For any given exposed server:
- T+0 minutes: Server starts with default configuration.
- T+hours to days: Automated scanners identify the open port.
- T+days to weeks: Attackers begin probing the API.
- T+ongoing: Resource abuse continues until detected through cost monitoring or performance degradation.
Detection typically occurs when cloud bills spike unexpectedly or when monitoring alerts fire on sustained high CPU usage.
Which Controls Failed or Were Missing
Network segmentation failed completely. The servers were accessible from any IP address on the internet. No firewall rules, security groups, or network policies restricted access to trusted networks.
Authentication was absent. Ollama's API lacks built-in authentication. Teams deployed it assuming the service would remain internal, without adding an authentication layer via reverse proxy or API gateway.
Default configuration was accepted without review. The 0.0.0.0 binding went unquestioned during deployment. No one asked, "Should this service be reachable from outside our network?"
Change management didn't catch the risk. If these deployments went through any approval process, reviewers didn't identify the security implications of an unauthenticated API with compute access.
Asset inventory was incomplete. Many organizations running exposed Ollama servers likely don't know they're running them. Shadow IT or developer experimentation led to production deployments without security review.
What the Relevant Standards Require
PCI DSS v4.0.1 Requirement 1.2.1 mandates restricting inbound and outbound traffic to that which is necessary for the cardholder data environment. An LLM API exposed to the entire internet fails this requirement. You need documented business justification for any service accessible from untrusted networks.
NIST 800-53 Rev 5 SC-7 (Boundary Protection) requires you to monitor and control communications at external boundaries and key internal boundaries. Ollama servers in cloud environments with no network controls violate this baseline. You must implement boundary protection mechanisms that deny network traffic by default and allow by exception.
ISO/IEC 27001:2022 Annex A.13.1.3 addresses network segregation. Running an inference API without network isolation between your AI infrastructure and public networks contradicts this control. You need to separate networks carrying different risk levels.
SOC 2 Type II CC6.6 requires logical access controls to prevent unauthorized access to data and systems. An API with no authentication mechanism can't meet this criterion. You must implement authentication and authorization before you can claim compliance.
For teams not bound by formal compliance, these requirements still represent sound engineering. If you're running compute resources that cost money and could potentially access sensitive data, you need access controls.
Lessons and Action Items for Your Team
Audit your LLM deployments now. Search your cloud accounts for instances running on port 11434. Check container registries for Ollama images. Ask your development teams what AI tools they're running. You can't secure what you don't know exists.
Implement network restrictions before authentication. Bind Ollama to 127.0.0.1 or a private network interface. Configure security groups or firewall rules to allow access only from your application servers or VPN. Network controls are your first line of defense.
Add authentication via reverse proxy. Deploy nginx, Caddy, or your API gateway in front of Ollama. Implement API key authentication, mutual TLS, or OAuth depending on your use case. The proxy handles auth; Ollama remains unchanged.
Treat LLM infrastructure like database servers. You wouldn't expose PostgreSQL to the internet without authentication. Apply the same thinking to Ollama. It's a stateful service with access to compute resources and potentially sensitive data in model context.
Update your deployment templates. If you're using Terraform, Kubernetes manifests, or Docker Compose files to deploy Ollama, embed network restrictions and authentication requirements in the template. Make secure deployment the default.
Monitor for unexpected API usage. Set up alerts for unusual request patterns, high token consumption, or requests from unexpected source IPs. Resource abuse often shows up in metrics before it appears in your cloud bill.
Document your AI infrastructure in your asset inventory. Track where you're running LLMs, what models you've deployed, and what data they can access. Include this in your regular security reviews.
The 175,000 exposed servers represent a gap between AI adoption speed and security maturity. Your team can close that gap by treating LLM infrastructure with the same rigor you apply to traditional application components. Network segmentation, authentication, and monitoring aren't optional extras. They're essential for any service handling requests and consuming resources.
