Skip to main content
  1. Blog/

LiteLLM Supply Chain Attack: 40 Minutes That Hit 500K Systems

ThreatNeuron
Author
ThreatNeuron
Attacks. Defenses. Everything in between.
Table of Contents

On March 27, 2026, two malicious versions of LiteLLM — a widely used Python library for managing LLM API calls — appeared on PyPI. They stayed live for roughly 40 minutes. In that window, the compromised packages harvested API keys, SSH keys, cloud credentials, and database secrets from every system that pulled the update, exfiltrating everything to a server at models.litellm[.]cloud. The estimated reach: around 500,000 machines.

Forty minutes doesn’t sound like much. But for a package with approximately 97 million monthly downloads — one that sits in over a third of surveyed cloud environments — it was more than enough. The LiteLLM supply chain attack has already triggered a class action lawsuit, forced multiple AI labs into emergency investigations, and exposed just how thin the trust layer is between the AI industry and the open-source packages it depends on.

How the Attack Chain Unfolded
#

This wasn’t a smash-and-grab on a single package registry. The attackers — a group tracked as TeamPCP — built their access through a multi-stage supply chain compromise that started weeks before anyone noticed.

The initial foothold came in late February 2026, when TeamPCP compromised Trivy, the popular open-source vulnerability scanner maintained by Aqua Security. The irony is hard to miss: a tool designed to find security problems became the vector for one. Malware injected into Trivy harvested CI/CD credentials — the keys to automated build and deployment pipelines.

Those stolen CI/CD credentials gave the attackers lateral movement into downstream projects. They pivoted into KICS (a static analysis tool maintained by Checkmarx) and, critically, into the LiteLLM build pipeline. From there, TeamPCP pushed versions 1.82.7 and 1.82.8 to PyPI with credential-stealing payloads baked in.

The attack timeline looked something like this:

  • Late February 2026 — Trivy compromised; CI/CD credentials stolen
  • March 2026 — Malware injected into Trivy’s build; lateral movement to KICS, LiteLLM, and Telnyx
  • March 27, 2026 — Malicious LiteLLM versions 1.82.7 and 1.82.8 published to PyPI
  • ~40 minutes later — Packages identified and pulled from PyPI
  • April 2, 2026 — Mercor, the highest-profile victim, publicly confirmed its breach

What makes this particularly nasty is the cascading nature. Compromising a vulnerability scanner gave the attackers access to security tooling CI/CD pipelines, which gave them access to developer tooling pipelines, which gave them access to production AI infrastructure. Each hop amplified the blast radius.

The Mercor Breach: Where Theory Meets Damage
#

Mercor, an AI recruiting startup valued at $10 billion after a $350 million Series C led by Felicis Ventures, became the most visible casualty. The company recruits specialists in medicine, law, and literature to produce AI training data for clients including Anthropic, OpenAI, and Meta. That client list made it a high-value target.

According to Fortune, TeamPCP planted the credential-harvesting code with the explicit goal of spreading across the AI industry’s infrastructure. And it worked. The extortion group Lapsus$, which partnered with TeamPCP along with ransomware gangs CipherForce and Vect, claims to have pulled roughly 4 terabytes of data from Mercor alone.

The stolen data reportedly includes:

  • 939 GB of source code and database records
  • 211 GB of user database information
  • ~3 TB of video interviews and identity documents from contractors
  • Slack messages, internal ticketing data, and recordings of AI system conversations
  • Datasets used by customers and details about undisclosed AI projects

That last bullet is the one keeping AI lab executives up at night. Multiple organizations — including Meta, OpenAI, Anthropic, and Google — launched investigations to determine whether proprietary training methodologies were exposed. The Next Web reported that Meta suspended its collaboration with Mercor indefinitely. Given that Meta had a $27 billion AI infrastructure deal in motion as of March 2026, the financial exposure here goes well beyond the breach itself.

Around 40,000 current and former Mercor contractors had personal information compromised. A class action lawsuit was filed in the Northern District of California on April 1, 2026.

Why AI Infrastructure Is a Prime Supply Chain Target
#

LiteLLM isn’t some obscure utility library. It’s a core piece of infrastructure for organizations running multi-model AI deployments — it abstracts the API differences between OpenAI, Anthropic, Cohere, and dozens of other providers into a single interface. That’s exactly what makes it attractive to attackers: one compromised dependency, and you’re harvesting credentials for every LLM provider an organization uses.

The broader pattern here is that AI infrastructure has a dependency problem that mirrors — and arguably exceeds — what the rest of the software industry faced with events like the SolarWinds attack and the event-stream npm compromise. A handful of open-source packages sit at the foundation of thousands of production AI systems, often installed with minimal verification and broad permissions.

The numbers tell the story: LiteLLM’s 97 million monthly downloads and presence in 36% of surveyed cloud environments mean a successful compromise propagates fast and wide. And unlike traditional software dependencies, AI infrastructure packages often handle credentials for expensive, high-value API services. Stealing an OpenAI API key isn’t just an authentication bypass — it’s direct access to compute resources that cost real money and training data that represents months of work.

This attack also highlighted a blind spot in how organizations think about AI-related threats. Most AI security conversations focus on prompt injection, model poisoning, or adversarial inputs. Supply chain compromise of the tooling around models gets far less attention, despite being a more straightforward path to mass credential theft.

The 40-Minute Window Problem
#

The malicious packages were live for approximately 40 minutes — and that was enough to hit an estimated 500,000 machines. According to The Register, over 1,000 SaaS environments were dealing with cascading effects in the aftermath.

This exposes a timing problem that the Python packaging ecosystem hasn’t solved. Automated dependency updates — whether through CI/CD pipelines, Docker builds, or development environments — can pull and install a compromised package within minutes of it hitting the registry. Pinning exact versions helps, but only if you’re not also running automated tools that bump those pins.

The detection-to-removal window matters enormously. PyPI’s maintainers acted fast once the compromise was flagged, but 40 minutes was already too late for hundreds of thousands of installations. This isn’t a criticism of PyPI specifically — no package registry has a reliable mechanism to prevent a maintainer-account-level compromise from reaching end users in near real-time.

Organizations that fared best in this incident shared a few common traits:

  • Dependency pinning with hash verification — not just version pins, but cryptographic hash locks that reject unexpected package contents
  • Private PyPI mirrors with staging delays — new package versions sat in a staging environment for hours before propagating to production
  • Runtime credential isolation — AI service API keys stored in secret managers with short-lived tokens, limiting the value of stolen credentials
  • Network egress monitoring — outbound connections to unexpected domains (like models.litellm[.]cloud) triggered alerts before data exfiltration completed

Teams without these controls were essentially trusting that every upstream maintainer account remained uncompromised at all times. That’s not a security posture — it’s a hope.

What Security Teams Should Do Now
#

If your organization uses LiteLLM — or any of the other packages caught in this supply chain — the immediate priority is credential rotation. Every API key, SSH key, cloud credential, and database password that existed on a system where versions 1.82.7 or 1.82.8 were installed should be considered compromised and rotated.

Beyond the immediate response, this incident should push security teams to rethink how they manage open-source AI dependencies:

Treat AI tooling dependencies like privileged software
#

LiteLLM handles credentials for your most expensive and sensitive AI services. It should go through the same vetting and monitoring as any other privileged component in your stack — not get quietly added as a transitive dependency that nobody reviews.

Implement supply chain verification at multiple layers
#

Package signing, hash pinning, and SLSA provenance verification aren’t just nice-to-haves. The SLSA framework provides a structured approach to supply chain integrity that would have caught the tampered build artifacts in this attack. If your CI/CD pipeline blindly pulls the latest version of anything from a public registry, you’re one compromised maintainer account away from a breach.

Monitor for anomalous package behavior
#

Tools like Socket analyze packages for suspicious behavior — unexpected network calls, credential access patterns, and obfuscated code — before they reach your systems. This kind of pre-installation analysis catches exactly the type of payload TeamPCP deployed.

Segment your AI infrastructure credentials
#

Don’t let a single compromised system yield credentials for every AI service you use. Separate API keys by environment, use short-lived tokens where providers support them, and enforce least-privilege access. If your staging environment’s LiteLLM instance gets popped, it shouldn’t have production API keys.

The traditional approach to AI threat detection tends to focus on model-layer attacks. This incident is a reminder that the most damaging AI security failures can happen in the build pipeline, not at inference time.

Key Takeaways
#

  1. The LiteLLM supply chain attack compromised an estimated 500,000 machines in roughly 40 minutes through malicious PyPI packages (versions 1.82.7 and 1.82.8) that harvested credentials and exfiltrated them to attacker-controlled infrastructure.

  2. The attack was multi-stage, originating from a compromise of the Trivy vulnerability scanner weeks earlier, with stolen CI/CD credentials enabling lateral movement into LiteLLM’s build pipeline.

  3. AI infrastructure packages are high-value supply chain targets because they handle credentials for expensive API services and sit in a large percentage of cloud environments — LiteLLM alone has 97 million monthly downloads.

  4. Detection speed doesn’t matter if propagation is instant. Forty minutes between publication and removal was enough for mass compromise. Organizations need pre-installation verification, not just fast takedowns.

  5. Credential rotation is the immediate action item for any organization that may have installed the affected versions. Every secret on those systems should be treated as compromised.

  6. Supply chain security for AI tooling deserves the same rigor as any other privileged software, including hash-pinned dependencies, private registry mirrors, SLSA provenance checks, and runtime egress monitoring.

Sources & References
#

Related