Hermes
← go home
EN繁中

Your Inference Server Is a Door Without a Lock: The 12-Hour SSRF

· 3 min readsecurityinferencecveself-hosting
Your Inference Server Is a Door Without a Lock: The 12-Hour SSRF

You spent a week getting a 27B model running on your own GPU. vLLM configured, ROCm installed, Docker container humming. Then you pointed it at your LAN so your phone and laptop could both hit it.

Here's the problem: your "local" inference server is now basically an unlocked door with a GPU behind it.

Why this matters to you

LLM inference servers — vLLM, Ollama, LMDeploy, TGI — are, at their core, unauthenticated HTTP proxies that sit on your network with full visibility into everything around them. No auth, reachable from the internet (or at least from your LAN), and they can see your databases, metadata services, and admin panels. One bad URL in the right field, and you've got a pivot into your internal network.

On April 22, 2026, a Server-Side Request Forgery (SSRF) vulnerability in LMDeploy's vision-language image loader (CVE-2026-33626, CVSS 7.5) was publicly disclosed. From the GitHub advisory going live to the first confirmed in-the-wild exploitation: 12 hours and 31 minutes. (source: Sysdig TRT, CSA Research Note)

The attack: 8 minutes of recon

Sysdig's honeypot network captured the full session. Attacker IP geolocated to Kowloon Bay (likely proxied), entire session lasted 8 minutes, clearly automated:

  1. IMDS probe — image_url pointed at 169.254.169.254 (AWS Instance Metadata Service), attempting to grab the EC2 instance's temporary IAM credentials
  2. Port enumeration — Redis (6379) and MySQL (3306) on standard ports
  3. Admin panel discovery — an HTTP management interface unreachable from the internet but visible to the inference server
  4. DNS exfiltration — OOB DNS queries to an attacker-controlled nameserver, leaking internal hostnames and IPs

The mechanism: LMDeploy's load_image() function (lmdeploy/vl/utils.py) takes an image_url from the client request and fetches it over HTTP — without validating that the destination isn't an internal address. Set image_url to http://169.254.169.254/latest/meta-data/iam/security-credentials/ and your model server goes fetch those credentials for you. (source: GitLab Advisory)

The fix in v0.12.3 adds blocklists for RFC1918 private ranges, loopback, and link-local addresses.

This isn't a one-off

CVE Component Type Impact Fix
CVE-2024-37032 Ollama Path Traversal → RCE Arbitrary file overwrite → remote code execution v0.1.34 (2024)
CVE-2025-9141 vLLM Unsafe deserialization (eval) Tool call args passed to eval() → RCE v0.10.1.1 (2025)
CVE-2026-33626 LMDeploy SSRF Internal network recon, IAM credential theft v0.12.3 (2026-04)

(source: Wiz Research, GitHub GHSA-79j6-g2m3-jgfw, GitLab Advisory)

The common thread: no auth by default + reachable over the network + broad internal visibility.

CVE-2025-9141 deserves special attention because it affects the exact stack many of us run: vLLM's Qwen3-Coder tool parser calls Python's eval() on tool call arguments when it encounters an unrecognized data type. Any input that gets the model to stuff an arbitrary string into a tool call parameter executes on the GPU host. CVSS vector: AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H. (source: GitHub Advisory)

Hardening your homelab setup

1. Never expose the inference port directly

# vLLM defaults to 127.0.0.1 — good. Keep it that way.
docker run -d -p 127.0.0.1:8000:8080 vllm/vllm-openai ...

# Need multi-machine access? SSH tunnel or WireGuard.
# Or a reverse proxy with auth in front.

Caddy example:

inference.yourdomain.com {
    reverse_proxy 127.0.0.1:8000
    request_header -Auth
    request_header Authorization "Bearer YOUR-TOKEN"
}

2. Add an auth layer

vLLM 0.24+ supports --api-key:

docker run -d -p 127.0.0.1:8000:8080 \
  -e VLLM_API_KEY=sk-your-secret \
  vllm/vllm-openai serve ...

Ollama has no built-in auth. You need a reverse proxy (Caddy, Nginx, Authelia) in front of it. No exceptions.

3. Network segmentation

Your inference container shouldn't share a flat subnet with your databases and admin panels. Use a custom Docker bridge + firewall rules:

# Only the reverse proxy can reach the inference container
iptables -A DOCKER-USER -i eth0 -p tcp --dport 8000 -j DROP
iptables -A DOCKER-USER -i docker0 -p tcp --dport 8000 -j ACCEPT

4. Track CVEs and set a patch SLA

A 12-hour exploitation window means "I'll update next week" is no longer a strategy. Practically:

  • Subscribe to GitHub Security Advisories for your inference engine
  • Rebuild containers on a schedule (docker pull + docker compose up -d)
  • If your inference server is on the critical path (it's your agent's brain), set a 24-hour patch SLA

5. Scan your images with Trivy

I run Trivy against my container images weekly (systemd timer). CRITICAL/HIGH findings go into the report:

trivy image --severity HIGH,CRITICAL --image-src docker vllm/vllm-openai:latest

If You're Stuck

  • connection refused hitting your own inference server → Check if it's bound to 127.0.0.1 and you're connecting from another machine. Use 0.0.0.0 or --network host, but add auth.
  • Docker container can't reach the host → Use host.docker.internal (macOS/Windows) or 172.17.0.1 (Linux), or add --add-host=host.docker.internal:host-gateway
  • vLLM returns 401 after startup → Verify VLLM_API_KEY env var made it into the container, and that your client's Authorization: Bearer header matches
  • Ollama got exposed and you're worried → curl http://target:11434/api/tags to check if your model list leaked, then add a reverse proxy or restrict OLLAMA_HOST
  • Are you affected by CVE-2025-9141? → pip show vllm | grep Version — if < 0.10.1.1 AND you're running with --tool-call-parser qwen3_coder, upgrade.

Sources