← Back to Archives

Squeezing the Balloon: Why Securing the Open Source Build Process is Only the Beginning

If you spend enough time studying physics, you start to view every complex system through the lens of energy conservation. When you apply pressure to a closed, incompressible system, the energy doesn't simply vanish; it displaces. It flows outward, probing the boundaries until it finds the next path of least resistance.

In the cybersecurity world, we call this dynamic "squeezing the balloon." When defenders successfully barricade one massive vulnerability, attackers don't simply give up—the pressure shifts, forcing them to exploit the next weakest point in the system.

Over the last couple of years, the open-source software community successfully squeezed a massive, highly vulnerable end of the balloon. We fundamentally changed how software is uploaded to the Internet's major package registries (like npm for JavaScript or PyPI for Python).

But as two recent security talks—William Enck’s NDSS keynote and Zach Steindler’s presentation at the USENIX Security Enigma Track—clearly demonstrate, the attackers didn't pack up and go home. They just moved to the parts of the balloon that are currently bulging out: the automated assembly lines (CI/CD pipelines) that build our code, and the massive data files (SBOMs and VEX) we use to track vulnerabilities.

Let's break down exactly what we solved, how the threat shifted, and what the security research community must do to keep the Internet's infrastructure from collapsing under its own weight.

Throwing Away the Master Key

To understand the victory, you have to understand how we used to secure open-source software.

For decades, if a developer wanted to publish an update to a widely used piece of software, they had to prove their identity using a long-lived secret—like a GPG key (a cryptographic signature tied to a person) or a permanent API password (a static token granting persistent upload access).

Think of this like a physical master key to a highly sensitive supply warehouse. The problem is that open-source maintainers are mostly unpaid volunteers. They are managing dozens of these critical "warehouses" from laptops in their living rooms. Eventually, a laptop gets compromised, a key gets accidentally committed to a public repository, or an attacker uses deceptive patterns to trick a maintainer into handing the key over. Once the attacker holds that master key, they can upload malicious software disguised as a legitimate update, instantly infecting thousands of downstream users who trust the warehouse.

At the USENIX Security Enigma Track in 2025, Zach Steindler detailed a massive, coordinated triumph to fix this (full disclosure: I worked with Zach on his talk as part of the Enigma Track Program Committee).

Instead of relying on permanent passwords, major registries adopted an architecture called Trusted Publishing (powered by identity protocols like OpenID Connect (OIDC) and transparency logs like Sigstore).

Here is how it works in practice: When the software is ready to be published, the automated build system essentially walks up to a digital bouncer. The bouncer checks a highly specific set of non-falsifiable credentials (e.g., "Did this request originate from the official repository? Was it triggered by an authorized GitHub Action?").

If everything checks out, the bouncer issues a temporary, single-use cryptographic key. The build system uses that ephemeral key to sign the software, proving exactly how and where it was built—creating a cryptographic receipt known as SLSA Build Provenance.

Then, milliseconds later, that temporary key expires and becomes entirely useless.

There are no permanent passwords for attackers to steal anymore. We successfully squeezed the balloon.

Hijacking the Assembly Line

This brings us to the Network and Distributed System Security Symposium (NDSS) 2026, where Professor William Enck delivered a sobering keynote address. He praised the adoption of Trusted Publishing, but he highlighted a terrifying new reality: because attackers can no longer steal developer keys to bypass the front door, they are now targeting the automated assembly line itself.

That assembly line is the CI/CD pipeline (Continuous Integration / Continuous Deployment). Services like GitHub Actions run these pipelines. They are essentially fleets of cloud-based robots that automatically test and package code whenever a developer proposes a change.

The fatal flaw of CI/CD pipelines is that they often mix trusted infrastructure with untrusted input.

In open source, anyone on the Internet can submit a proposed code change (a "Pull Request"). To an attacker, a Pull Request isn't just a suggestion; it's a vehicle to inject poison into the assembly line. Because CI/CD robots are programmed to automatically grab new code and test it, an attacker can trick the robots into executing malicious commands during the test.

We've seen this in recent, highly sophisticated compromises targeting major frameworks like Angular and TanStack. Attackers use a few specific techniques:

  • "Pwn Requests": Attackers exploit misconfigured settings (like using pull_request_target in GitHub Actions) that grant too much administrative privilege to a random user's code submission. Because this setting runs the untrusted code inside the secure boundary of the main repository, the robots run the attacker's code with elevated permissions. The attacker uses that access to steal internal secrets or rewrite the software from the inside.
  • Zombie Workflows: Attackers find old, forgotten automation scripts lingering on historical branches in a repository. Even if the maintainer patched the vulnerability on the main branch, attackers can trigger the outdated script on the old branch to execute legacy vulnerabilities.
  • Cache Blasting: To speed things up, build robots save (or "cache") certain files for later use. Attackers can intentionally submit junk code designed to overwrite that temporary memory with malicious files. When the legitimate developer runs the next build, the robot pulls the poisoned files from the cache, infecting the official release.

Automating the Noise

Even if we perfectly secure the CI/CD assembly line, Enck pointed out a secondary, equally massive problem. The data we use to track software security is a mess.

When a package is built, we increasingly generate a Software Bill of Materials (SBOM). It’s exactly what it sounds like: a digital ingredient list.

The problem is that modern software is incredibly complex. If your ingredient list says "Contains Peanuts," automated security scanners will sound a massive alarm because peanuts are a known allergen (a known vulnerability). But what if the "peanuts" are shrink-wrapped in impenetrable plastic, or only used in a standalone testing module that never actually gets shipped to the end user? The automated alarms still blare, creating paralyzing alert fatigue.

To fix this, the industry created VEX (Vulnerability Exploitability eXchange). A VEX is basically a certified sticky note attached to the ingredient list that says, "Yes, we contain peanuts, but I cryptographically promise they are isolated in plastic and cannot hurt you."

The glaring issue? Humans are currently writing those VEX statements. Humans make mistakes. Sometimes they get tired of the alarms and just write a VEX statement to make the warnings go away without truly verifying the risk.

We cannot rely on humans to manually verify millions of lines of code. We need concerted, automated solutions from the research and development community.

Recent research, like the CHAINFUZZ paper presented by Deng et al. (Fudan University) at USENIX Security '25, shows us the way forward. CHAINFUZZ uses a technique called Directed Greybox Fuzzing. Instead of a human guessing if a vulnerability is dangerous, the system mathematically attacks its own code. It automatically throws millions of mutated inputs at the software to see if there is any physical execution path for an attacker to actually reach the vulnerable "peanut."

If the system proves that the peanut is completely unreachable, it automatically generates a mathematically sound "Not Affected" VEX statement. No human error. No guesswork.

The Human Toll

And that human element is exactly why this matters.

The most critical lessons learned from Enck's and Steindler's talks aren't strictly about cryptography or code execution; they are about cognitive bandwidth.

Open-source maintainers are burning out at a catastrophic rate. They are being crushed by the uncompensated administrative burdens of compliance mandates—like the EU's Cyber Resilience Act (CRA), which forces 'Open-Source Software Stewards' into strict 24-hour vulnerability reporting and documentation regimes just to keep their projects viable for downstream users. (See the FreeBSD Foundation and Eclipse Foundation on these effects.)

Simultaneously, they are flooded with AI-generated bug reports—a crisis that burns out maintainers whether the AI is hallucinating or working perfectly. When the tools hallucinate, maintainers drown in 'AI slop.' For instance, Daniel Stenberg was forced to suspend the cURL bug bounty program earlier this year because his team was overwhelmed by confident, yet completely fabricated, vulnerability reports. But even when the AI works exactly as intended, the result is still cognitive collapse. Just last week, Linus Torvalds declared the Linux kernel security mailing list 'almost entirely unmanageable'. Why? Because AI tools are successfully finding real vulnerabilities, prompting hundreds of 'drive-by' researchers to spam the mailing list with the exact same automated discoveries without ever bothering to write a patch. Whether it is hallucinated slop or duplicated spam, the result is the same: a total collapse of maintainer cognitive bandwidth.

When the underlying infrastructure of the Internet relies on the unpaid weekends of a few thousand developers, we cannot ask them to manually defend complex CI/CD pipelines or write endless VEX statements.

The package registries did their part by throwing away the master key. Now, it is up to the security research community to build the automated, mathematically verified tools that will protect our assembly lines, filter the noise, and let human developers get back to doing what they do best: building the future.