TL;DR: Our deep dive into open source projects’ CI/CD systems has revealed that build pipelines can be just as vulnerable as any other link in the software supply chain. We found hundreds of zero days on open source projects’ build pipelines with our detection at scale and responsibly disclosed them. Jump to the Research at Scale section to learn more.

Introduction

When discussing software supply chain security, the conversation often revolves around known vulnerabilities (CVEs), Software Bills of Materials (SBOMs), and the diligence required to manage open source dependencies. These are indeed crucial. However, there is a hidden layer that many organizations—and even open source projects—often overlook: the build pipeline itself.

Open source communities have grown accustomed to scanning for malicious packages (e.g., typo-squatting, dependency confusion) and known CVEs, which are indeed mature areas of supply chain security. But few have turned their attention to the scripts, tools, and CI/CD workflows that piece everything together. Our research has shown that a large number of real vulnerabilities—what we call zero-days in the build pipeline—can be discovered by looking more closely at these pipelines.

This post is a deep dive into how we uncovered several previously unknown vulnerabilities in open source build pipelines at scale, the broader threat model, and the practical countermeasures you can use to protect yourself.

The Current State of Supply Chain Security

Mature Areas of Supply Chain Security

• Known Vulnerabilities (CVEs): Tools like Software Composition Analysis (SCA) scanners and SBOM generators have made it possible to map out dependencies, flagging insecure or outdated libraries in code.
• Malicious Packages: We’ve seen attacks on package repositories like NPM and PyPI through typo-squatting, dependency confusion, and credential theft. The open source ecosystem has improved its defenses against these vectors, though they remain a concern.

Overlooked Attack Surfaces

• Build Pipelines: These are often treated simply as “plumbing” to compile, package, and test code. In reality, because they execute code and can deploy artifacts to production or to public repositories, a compromised pipeline can have devastating consequences.
• Tooling Gaps: Few tools exist to create SBOMs of build-time dependencies. Popular scanning solutions typically inventory only what ends up in the final artifact. This leaves the ephemeral scripts, Docker images, and third-party actions used exclusively at build time largely unexamined.

Why the Build Pipeline Matters

Threat Modeling with SLSA

The Open Source Security Foundation’s SLSA framework (Supply Chain Levels for Software Artifacts) highlights the interactions between developers, version control systems, build pipelines, and artifact registries. A malicious actor targeting the build system can replace or modify artifacts so that, from the consumer’s perspective, everything looks normal. This is precisely what happened during the SolarWinds incident, where an attacker slipped malicious code into a trusted commercial product’s update.

GitHub Actions: A Prime Target

• Popularity: Many open source projects on GitHub rely on GitHub Actions because it is free for public repositories.
• Risky Event Triggers: Certain triggers—like pull_request_target, workflow_run, or even issue_comment—can inadvertently grant external contributors the ability to run privileged workflows.
• Default Permissions: Unless explicitly changed, GitHub Actions often come with default read-write permissions and automatically injected secrets, such as the GITHUB_TOKEN.
• Untrusted Code Execution: Pull Requests from forks are a core part of open source collaboration, but if a workflow is not well-hardened, an attacker can slip malicious commands into the build script.

In short, if the pipeline handling privileged credentials is manipulated, the attacker may pivot to exfiltrate secrets or publish compromised artifacts.

Real-World Attack Scenarios

A recurring pattern dubbed “Pwn Request” (playfully named after “Pull Request”) involves these key elements:

1. Public Repo with Risky Workflow Events
   A repository’s CI triggers on events like pull_request_target, which is particularly risky because it can expose secrets to a forked pull request if not carefully configured.
   
   
2. Injection Vulnerabilities and Living Off The Pipeline(lotp)
   Bash and JavaScript commands are prime infiltration points. Attackers exploit variables like github.head_ref or similar user-controlled inputs to achieve command injection. Also, some build tools (e.g., template engines, code-generation tools) which were never designed to accept untrusted code from outside contributors, can be exploited. We call those Living Off The Pipeline tools.
   
   
3. Privilege Escalation and Artifact Tampering
   Upon gaining remote code execution (RCE), an attacker can pivot to exfiltrate secrets or directly publish a backdoored artifact under the project’s name—without merging any malicious commits into the main branch.

A stark illustration is the risk around Terraform modules or GitHub Actions themselves. Many popular Terraform modules auto-publish to a registry whenever a specific workflow sees a new version tag. With trivial injection, an attacker can produce a backdoored artifact that unsuspecting users will pull directly into production infrastructure.

Research at Scale: The “Package Supply” Project

To verify these insights across tens of thousands of repositories, we created an internal system—Package Supply—to:

1. Ingest Large Sets of GitHub Repositories
   We gathered hundreds of thousands of open source projects, especially focusing on GitHub Actions.
   
   
2. Statically Analyze CI Workflows
   We scanned .github/workflows/*.yml files, looking for signs of injection, untrusted code checkout, risky events, and potential misuse of secrets.
   
   
3. Build a Dependency Graph
   Our system loaded data into a graph database so we could traverse the chain of dependencies. If a single action was vulnerable, its presence in many other repositories multiplied the potential “blast radius.”
   
   
4. Responsible Disclosure
   After identifying genuine zero-days and vulnerabilities, we disclosed them to project maintainers. In many cases, the maintainers were surprised at how trivial it was to exploit their pipeline.
   
   

This methodology led us to discover—and responsibly disclose—hundreds of build-pipeline zero-days across a variety of projects, from large infrastructure modules to popular security scanners. We have only scratched the surface as there are hundreds more we haven’t had time to triage and by refining our detections we will uncover even more.

Building Defenses: How to Find and Fix Pipeline Vulnerabilities

Below are practical tips, mostly centered on GitHub Actions, but applicable to other CI systems as well.

Inventory and Scan Your Build Pipelines

• Use Static Analysis Tools: We created and open-sourced poutine, a specialized SAST tool to quickly inspect build workflow files for injection points, untrusted code execution, and known-vulnerable actions. Learn more here.
• Harden Org-Level GitHub Actions configuration: Restrict allowed github actions in your org, set workflow permissions of the GITHUB_TOKEN to read only, disallow repos from creating self-hosted runners and require approval to run workflows from forks and outside collaborators.

Lock Down Permissions

• Set Explicit Permissions: By default, GITHUB_TOKEN may have read-write access. Override this by specifying the minimum required permissions in your workflow starting with none and adding only what is required:

permissions: {}

permissions:

  contents: read

  pull-requests: write

• If a job does not need to write to the repo, remove the write permission.
  
  
• Segregate Secrets: Use environment secrets or manual approvals to protect production credentials. Avoid dynamically loading secrets from a dictionary object, as this can expose them all in memory if one step is compromised.
  
  
• Avoid Personal Access Tokens: Rely on ephemeral tokens and OpenID Connect (OIDC) for cloud deployment credentials, restricting them to trusted branches or environments only.
  
  

Cautious Use of Risky Events

• pull_request vs. pull_request_target: If you do not absolutely need the features of pull_request_target (which exposes Build Time secrets even in the context of a PR from a fork), switch to the safer pull_request event.
• Avoid Script Interpolation: Instead of dropping variables directly into a Bash command, pass them as parameters or environment variables to limit injection risk.

Self-Hosted Runners

• Disable for Public Repos: Self-hosted runners can be extremely powerful but pose a big risk if an untrusted contributor triggers them. If you must use them, ensure ephemeral runners and segmented infrastructure so that each run is on a clean machine without broad network privileges.

Runtime Protections

• Harden the Runner: Tools such as Step Security’s Harden Runner provide real-time monitoring and blocking of malicious commands. They essentially insert a policy layer to watch for suspicious network or system calls.

Conclusion

Our deep dive into open source projects’ CI/CD systems has revealed that build pipelines can be just as vulnerable as any other link in the software supply chain. In an era when so many critical open source dependencies are maintained by volunteers or small teams, it is crucial for the broader community to take CI security every bit as seriously as production system security.

By combining threat modeling frameworks like SLSA, adopting least-privilege and secure defaults, and using specialized scanning tools such as poutine, developers and security teams can shore up the underappreciated build layer. This not only protects your own projects but also fortifies the shared open source ecosystem that underpins modern software.

For more detailed breakdowns, demos, and real disclosures, check out our write-ups on the Boost Security blog. We look forward to collaboration with the wider community in building a safer and more transparent open source supply chain.

Resources & Further Reading

• SLSA Framework
• OWASP CI/CD Top 10 Security Guidance
• poutine – An Open Source Build Pipeline Security Scanner
• messypoutine CTF for learning hands-on pipeline exploitation.