SOFTWARE SUPPLY CHAIN
Trying to identify spoofing in GitHub? May the 4th be with you!
May 3, 2023
“Commit spoofing” refers to the surprisingly easy process of committing code to a Github repository as someone else. Threat actors both internal and external to organizations have used this method to compromise the software supply chain with malicious code or even outsource work in bad faith. While there are technical methods to confirm an authors’ identity and raise visibility into unverified commits, which we will describe in detail in this article, they can be difficult to implement and therefore have low adoption. Arnica can help organizations solve this problem with developer anomaly detection, which brings 100% integration in one-click.
Source code management tools are designed to manage code changes and create an audit trail of contributors. Using events such as commits and code merges, GitHub allows developers to easily tell which code has been introduced or altered. GitHub also logs the user responsible for the change. This functionality is central to version control and ownership of code, but it’s surprisingly easy to push a commit to a GitHub project and make it appear as though it was committed by someone else. This vulnerability is called “commit spoofing,” and it makes it difficult to be sure that code commits are authentic because the named author of a commit in the commit log may not be the actual author.
In recent years, this tactic has been exploited by malicious actors to compromise open-source libraries that numerous codebases depend upon, as well as by developers inside an organization to outsource their work in bad faith.
In this article we will describe how easy it can be to make a commit appear as if it were authored by a trusted party. We will also present technical methods to confirm an author’s identity and raise the visibility of unverified commits, as well challenges with these methods. Finally, we will talk about how Arnica, with 100% coverage from day 1, can help organizations solve this problem by detecting anomalous developer behavior.
By default, when a developer configures their Git client with their name and email, there is no further verification of that identity when the commit is written and pushed. In fact, on a clean install of Git, the default configuration anonymizes the name and email by design. Unless this default is proactively changed, the following is what will show up in the name and user fields of the commit history:
It is very simple for anyone to change these values on the fly, which would result in commits that appear to have been authored by a certain identity, but in fact were not authored by them at all.
Let’s use an example to illustrate this point. R2D2 is a developer who works for Rebel Corp. and has decided to configure one set of commits to be attributed properly:
In an instant, R2D2 can then switch the attribution and commit code that appears to be authored by someone else. Perhaps R2D2 has decided that it would be a good idea to commit a function on behalf of a colleague/nemesis. Let's call him Death Star Admin:
Both of these commits were written by R2D2 on a single workstation, but without any further verification or monitoring, it will appear as if two different developers made two different commits:
Even with SSH keys enabled, which will prompt for authentication when pulling or pushing to the remote, this vulnerability of commit spoofing will still exist. This is because the SSH protocol is being used to validate credentials that give the user the ability to push or pull to the repository, but the scope of the authorization stops there. There is no further validation that ensures the commit author’s identity matches what has been written to the name and email fields of the commits themselves.
Let’s explore how this vulnerability can be critically harmful to organizations.
There have been several large-scale examples of how commit spoofing is exploited in the wild. From malicious actors attempting supply chain attacks to disgruntled former employees lashing out, and even developers attempting to secretly outsource their work.
One famous example of an internal employee fraudulently outsourcing his coding happened at Verizon, the American telecommunications company. The employee paid an engineer in China 20% of his salary to commit code on his behalf.
More recently, it was discovered that malicious actors have been masquerading as reputable open-source contributors to trick developers into thinking the code is trustworthy. These actors only need to know the name and email address of someone who is highly respected in the community—which can be easily accessed in legitimate repositories—and begin a campaign of spoofing to introduce harmful code that can appear to be written by a legitimate author.
Arnica also reported on a widespread malware attack on Github, where a researcher known as “Pl0x” created backdoor vulnerabilities in popular repos. One key strategy was to use commit spoofing to disguise malicious code as unverified commits by reputable authors. Users unknowingly downloaded compromised clones which executed code that uploaded environment variables (which often contain sensitive secrets).
The default behavior of both Git and the way commits are displayed on GitHub leaves many repositories vulnerable to attack or exploitation, especially if owners are not aware of this danger. However, there are some tools available that, if implemented correctly, can lower the chances that a malicious actor can get away with commit spoofing.
The first method is to use commit verification, or signed commits, which implements a cryptographic signature to sign each commit made in your local development environment. The key for your signature that allows GitHub to verify your commit is registered in your GitHub account. The end result is a clear “Verified” badge that will appear next to your commits; this can distinguish commits actually authored by you from commits that were made without your signature.
Building on the earlier example of R2D2 attempting to author a commit on behalf of a Death Star admin, let’s say the Death Star implemented a policy requiring GPG keys to sign commits. After R2D2 writes the fraudulent commit in the admin’s name, the admin then pushes a legitimate commit. They would look like this side by side:
The commit titled “Shut down trash compactors on the detention level” from the previous example is R2D2 committing code using the Git config name and email that matches the Death Star admin’s identity. However, the commit titled “My signature proves I wrote this” is the Death Star admin committing code from a workstation that has a cryptographically signed GPG key-pair set up, which GitHub recognizes when the signed commit is pushed to the remote. As a result, the Death Star admin’s authentic commit shows a green “Verified” badge, while R2D2’s does not.
Unfortunately, turning on the Verified badge in GitHub is not a straightforward process. There are several steps involved to be able to sign your commits:
Once you follow these steps, you are ready to sign your commits using this procedure.
GitHub also supports SSH keys and S/MIME to sign commits.
SSH keys are easier to generate and configure than GPG keys, and you can even use the same SSH key you use for remote authorization as your commit signature key. However, GPG keys can be revoked, can have an expiration date, and can also be marked as compromised, while SSH keys lack these features. For this reason, GitHub recommends that most use GPG keys.
S/MIME is usually implemented within the context of an organization, using the X.509 format issued by a certified authority arranged by the organization. GitHub provides instructions on how to configure Git and sign commits using S/MIME and does not require a public key to be associated with the account.
GitHub builds upon this “Verified” badge idea by also offering a setting called Vigilant mode. By default, all commits signed by a valid GPG, SSH, or S/MIME key are displayed with a “Verified” badge. Any commit signed by an invalid key is marked as “Unverified.” However, by default, all other commits that are left unsigned have no badge at all.
Vigilant mode, when enabled, will add a badge to all commits regardless of whether they are signed or not:
In addition to adding “Unverified” to unsigned commits, GitHub will also mark any commits that are signed and valid, but authored by someone other than the committer, as “Partially verified.” This indicates that the commit was not authored by the committer and may or may not have had the author’s consent.
By enabling Vigilant mode, it becomes more visually apparent which commits can be trusted and which commits should be treated with more scrutiny.
GitHub additionally offers Branch Protection, a feature that allows a repository owner to add rules and restrictions to activity on particular branches. One of the restrictions is the ability to require signed commits. If this option is enabled, then only commits that are signed and verified can be pushed to the branch. Partially verified commits are also allowed.
GPG authentication is difficult to implement on a single workstation, let alone many or all. The sheer number of steps required, as well as some of the complexities associated with those steps, makes it a cumbersome solution. And as a result, the likelihood of achieving 100% coverage across your development ecosystem is slim to none.
Some pitfalls that may trip up developers include the fact that GPG is not installed by default on Windows, Mac, or many Linux distributions, so there is a lot of variability when it comes to troubleshooting depending on the OS and version of the software being installed. For example, GitHub warns that some Linux installations will require a specific format of a GPG key, which will require an additional setting in Git. These kinds of complexities can lead to frustration for a developer setting GPG up for the first time.
There are also many steps and decisions a developer has to make when generating, verifying existing GPG keys, or renewing keys. GitHub supports a subset of GPG algorithms, and developers need to specify key size, type of key, length of validity, etc. This can add even more cognitive overhead to the process. Developers must also be careful when renewing expired keys, and ensure that they are renewing and re-uploading the same GPG key if they have used it to verify commits in the past. One mistake is to delete a previous expired GPG key and upload a new one, which could cause past commits to be permanently unverified. Managing or renewing previous and expired GPG keys is another headache of this approach because there are many details that the developer must keep in mind in order to do it correctly.
And finally, the annoyance of having to enter a passphrase associated with a GPG key each time a commit is signed is yet another layer of complexity. This is a familiar and easily solved problem when cloning, pushing, or pulling to GitHub using the SSH protocol because ssh-agent has strong integration with macOS Keychain. Unfortunately, the same cannot be said for gpg-agent, which can require additional software or steps to cache the password.
With Arnica’s anomaly detection solution, your organization can achieve 100% coverage across developers without requiring any of them to perform manual steps to sign commits. By continuously monitoring new code commits and developer actions, Arnica can detect anomalous activity and automatically take actions based on configurable policies and risk tolerances.
Arnica also integrates with existing collaboration tools such as Slack to provide a seamless experience for developers when verification is needed. For example, upon detection of a commit that was submitted at an unusual time or location, Arnica can prompt the developer to easily verify that the activity was authentic via Slack.
By lowering the barrier to entry for verification, both developers and the organization can be protected from the threat of commit spoofing with minimal disruption to development workflows.
.png)