BoostSecurity Blog

SLSA dip — At the Source of the problem!

Written by BoostSecurity.io | Nov 18, 2022 9:56:28 PM

This article is part of a series about the security of the software supply chain. Each article will be analyzing a component of the Supply chain Levels for Software Artifacts (SLSA) model in depth, from the developer’s workstation all the way to the consumer side of the chain.

 From the SLSA version 0.1 specification

This article is focused on Source Control Management (SCM). Nowadays this is almost synonymous with Git, and for many organizations, that means GitHub. Of course, there are many more great options to store, version, and collaboratively edit source code. Some organizations might have stricter requirements and want to store their code within their perimeter. To that end, GitHub offers an on-premise variant of GitHub Enterprise and GitHub Enterprise Server, but there is also GitLab, BitBucket, Gitea, and many others.

We will initially focus on studying the different strategies for attacking GitHub. You should consider this to be a living article that is a work in progress and will continue to evolve. We will first look at the SCM from a Red Team / Attackers’ perspective and then from a Blue Team / Defender’s perspective. Finally, we will combine all those attacks and mitigations into an attack tree built using Deciduous, an open-source security decision tree tool.

Red Team

To map out the GitHub Enterprise Cloud’s attack surface, we will use GitHub’s documentation. We’ve analyzed the attack surface relevant when considering all aspects of securing access to the source code. You will find all the links in the Appendix of this article.

We regroup all attacks by focusing on three malicious end goals (largely inspired by the SLSA threats catalog):

  • Submit malicious source code
  • Delete source code
  • Push a release tag pointing to vulnerable commit

There are numerous ways to achieve those end goals. The attack scenarios can range from relatively simple to highly complex. If we consider an insider threat, there are even more scenarios, and their mitigations might be more difficult to implement, especially for employees with administrative access.

Initial access

All the following attacks start with one of the following levels of access.

  • Insider threat
    – Who has write access to the repository.
    – Who has administrative access to the repository (this includes being a repository administrator or an organization owner).
  • External threat (with no repository access)
    – Compromise organization’s member’s accounts via phishing, credential reuse, leaked personal access token, etc. One of the easiest and potentially most powerful methods would be to find a leaked Personal Access Token with elevated permissions, such as “repo” (classic) or “read/write Contents” (fine-grained). Find them by hunting for them in dot-files or in CI manifests, where they can be used to grant CI more access than by default.
    – Compromise a vulnerable GitHub App installed on a target organization that has elevated permissions such as repository contents write access.
    – Steal SSH private keys such as a user’s key or a deploy key with write access

Submit malicious source code

— Directly, without review

If there is no Branch Protection enforced on the target branch or it has a weak configuration that does not even enforce peer review, the attack will be successful.

— Review one’s own changes via a sock puppet account

In this attack, the attacker manages to convince an Org Owner to grant write access to the attacker’s GitHub account (potentially repeating this more than once to have multiple fake identities). Using this account, the attacker then submits malicious code by opening a Pull Request and approving their changes before anyone else notices. With remote work being increasingly common, and new employees joining companies without ever seeing their colleagues in person, this is an increasingly plausible scenario.

— Use a robot account to submit changes

If the target organization was created before January 2022, the default settings for GitHub Actions grants the automatic token (GITHUB_TOKEN) the ability to approve pull requests. This allows an attacker to add a malicious GitHub Actions workflow alongside their malicious code changeset, which would be triggered when opening the pull request and could be set to approve the Pull Request immediately.

— Modify code after review

After the attacker submits a valid and good code change that is approved, the attacker abuses their existing approval to make further changes that include bad code while retaining the stale approval.

Another scenario is that the attacker could first be a good samaritan and approve the code of a fellow developer, let’s assume it’s a good code change, but it doesn’t matter. What matters is that once they have approved that pull request, they could abuse their own write access, add bad code and self-approve their own code change.

— Submit a change that is unreviewable

The attacker could submit their bad code as part of a huge changeset (with thousands of lines of code modified) and a lazy reviewer could conceivably review quickly and miss the bad code (i.e. LGTM).

Another option is for the attacker to add a new dependency that they in fact have full control over to a package manager manifest (such as package.json / package-lock.json, etc.). Let’s say they create a malicious library and submit it to NPM, unless the reviewer validates the new dependency they could sneak under the radar.

— Compromise another account

By compromising another account, they can achieve the same thing as with a sock puppet account, it’s just that this would be a legitimate, existing and potentially trusted senior employee account. Abusing the existing trust in this employee, others might not question the suggested changes as much.

— Trick reviewer into approving bad code

Without necessarily being a large code change, the attacker could make their bad code change none obvious to the reviewer. This could either be a vulnerability class that the reviewer is not familiar with or the code logic could be confusing enough that it’s not clear how a vulnerable condition could be reached.

— Reviewer blindly approves changes

The attacker might just have a lucky break and find a lazy reviewer that has a tendency to review things very sloppily.

— Project owner bypasses or disables controls

Assuming the attacker is a malicious insider with repository admin privileges or Org Owner, they could bypass the Branch Protection rule.

Another option is for them to temporarily disable the Branch Protection rule (or its enforcement on administrators).

Delete malicious source code

Once an attacker has managed to submit their malicious code changes, they might reasonably want to attempt to hide their tracks by removing it from the source control history.

Assuming there is reasonably robust Branch Protection configuration in place, force pushing on or deleting a protected branch should only be possible for administrators. The evil administrator would need to at least weaken the Branch Protection temporarily.

Push a release tag pointing to vulnerable commit

An attacker might be aware that the source code repository already contains a commit with a vulnerability they would be interested in exploiting if it were deployed to a production environment. Here we will make the assumption that the Build system is configured to build when a new tag with a certain pattern (ex. a semantic version like v1.2.3) is pushed. The attacker who otherwise would not be able to push code to the default branch, because it is subject to Branch Protection, but might still be able to push a remote tag that matches the expected pattern pointing to a malicious commit, which could also be in a branch that is not reviewed. On GitHub, by default, pushing tags simply requires repository write access.

Blue Team

In this section we will attempt to suggest mitigations to thwart the previously documented attacks or to minimize their impact.

Initial access

— Insider threat

  • Organization member with repository write access
    – The best mitigation is to set up as robust Branch Protection as possible. See the mitigation section for Directly submit without review below.
  • Repository administrators or Organization owners
    – As we could demonstrate in the Red Team section, there are numerous attacks that can be mounted by evil administrators. So really the only robust way to address this is to eliminate administrator accounts as much as possible and instead configure GitHub Organization and Repository settings as-code, using the GitHub terraform provider. Another option would be to create an internal (private) GitHub App which would help to make changes, without directly granting the permissions to the account.

— External threat

  • Compromise Org member account (phishing, credential reuse, personal access token leakage, etc.)
    – The consequences of a leaked Personal Access Token with high privileges can be disastrous, so the best mitigation strategy is to use the Fine-grained Personal Access Tokens and configure them with least privileges in mind, also setting an expiration policy forcing you to rotate them from time to time, as well as run audits of their usage. Fine-grained Personal Access Tokens can also be subject to explicit approval by Organization Owner and audited.
    – There is no way to directly enforce a strong password policy for normal accounts, but as an organization owner you can at least enforce 2FA. That is definitely a must have, but given that GitHub still allows SMS and TOTP, there is no way to know or enforce if your members use phishing resistant WebAuthn (U2F) tokens.
    – With GitHub Enterprise, you could enforce SSO to your IdP using SAML or OpenID Connect.
    – Using more expensive GitHub Enterprise pricing plans you could even consider using Enterprise Managed Accounts.
  • Compromise a vulnerable GitHub App installed on target organization which has elevated permissions (such as repository contents write)
    – The general recommendation is to avoid installing GitHub Apps (or OAuth apps), especially if they request elevated permissions (such as Contents write), but if you have to limit to highly trusted vendors.
  • Steal SSH private key (can be a user’s key or a deploy key with write access)
    – The recommendation is to train all developers to use an SSH agent with a strong passphrase and ideally not have it unlocked all the time. Depending on your Operating System, there may be support for smart cards, or secure enclaves with biometric authentication (such as Secretive on macOS). This is nearly impossible to enforce from an Organization’s perspective, but enforcing signed commits in your Branch Protection, might make simply stealing the SSH key not as useful. Protecting your GPG (or SSH signing key) is also just as important (smart cards or secure enclaves options are also great here).

Submit malicious source code

— Directly submit without review

The best mitigation is to set up the most robust Branch Protection possible.

Here is our recommendation:

  • Require a pull request before merging
    – Require approvals
    – Dismiss stale pull request approvals when new commits are pushed
    – Require review from Code Owners
    – Allow specified actors to bypass required pull requests (avoid unless you absolutely need to)
    – Require approval of the most recent push (this is a new setting, as of October 2022, and is really great mitigation for some of our attack scenarios)
    – Require status checks to pass before merging (it you have some form of CI with tests, linters, SAST, it would be great to enforce those)
    – Require signed commits (this is great for end-to-end accountability)
    – Enforce Branch Protection for administrator (i.e. “Do not allow bypassing the above settings”)
  • DO NOT set the following settings
    – Allow force pushes
    – Allow deletions

— Review own change through a sock puppet account

The rule of thumb is to have a single organization membership per employee. This is difficult to do with normal GitHub accounts and that’s where enforcing SSO to your IdP using SAML or OpenID Connect might help.

— Use a robot account to submit change

Review your Organization setting for GitHub Actions and make sure that the Workflow permissions are set to prevent GitHub Actions from creating or approving pull requests using the automatic token.

You can find this at the bottom of the GitHub Actions settings of your organization

— Modify code after review

  • Attacker submits good code, gets approval, then submits bad code
    – The mitigation is to set your Branch Protection to “Dismiss stale pull request approvals when new commits are pushed”.
  • Attacker approves someone else’s good code, then submits bad code and self-approves changes
    – The mitigation is to set your Branch Protection to “Require approval of the most recent push”.

— Submit a change that is unreviewable

  • Attacker submits bad code as part of huge changeset
    – Only accept pull requests with small, manageable changesets.
  • Attacker adds evil dependency
    – Only accept trusted dependencies.

— Compromise another account

Following the recommendations in the Initial Access section should help limit the risk of compromising other accounts, but there is no silver bullet.

— Trick reviewer into approving bad code

While it’s not a bad idea to use SAST might help the reviewer to identify vulnerabilities, but the attacker would reasonably test that SAST would not detect their bad code. Enforcing a Two-person review Branch Protection rule might lower that risk even more, but ultimately if the attacker is really trying to fool you, they might succeed.

— Reviewer blindly approves changes

The attacker might carefully pick their reviewer, if they know they tend to be straight shooters that LGTM a little too fast, otherwise they might just get lucky and submit pull requests when people are tired. There is no good solution to this other than maybe audit pull request reviews, see those who tend to approve a little too fast and remind them to be more vigilant.

— Project owner bypasses or disables controls

As discussed in the Initial Access > Insider Threat section, the best option is to limit the number of administrators and use configuration as code.

Delete malicious source code

Assuming you have Branch Protection, you should never set the Allow force pushes or Allow deletions options. They are off by default when you create a Branch Protection, leave them unchecked.

Otherwise, for administrators, as discussed in the Initial Access > Insider Threat section, the best option is to limit the number of administrators and use configuration as code.

Modify release tag to point to vulnerable commit

Use Tag Protection Rules to prevent normal users to push remote tags that match a certain sensitive pattern.

Otherwise, for administrators, as discussed in the Initial Access > Insider Threat section, the best option is to limit the number of administrators and use configuration as code.

Attack tree

In this section we combine all the attacks and mitigations presented in the previous sections into an attack tree. We’ve used the amazing Deciduous tool to create it and we are happy to share it in this GitHub repository. Keep in mind that this is a living article and we plan on updating the attack tree as new techniques are discovered. We welcome community contributions.

 Software Supply Chain Attack Tree — Source Control Management

Appendix: GitHub Enterprise Cloud attack surface analysis

This section contains numerous links back to GitHub’s documentation where we can find information about every single feature that could be abused or be used to harden the default configuration.
View full post