If your codecov site is hacked, there are a few things you can do to protect yourself. First, make sure you have updated your security software and that your passwords are strong. Second, be sure to keep all of your important files safe and secure. Finally, be sure to report any suspicious activity to your hosting company or the police.

Codecov was hacked in a way that impacts all of its customers. Thousands of commercial enterprises and open-source projects are affected. Here’s what you need to do if you’re one of them.

What is Codecov?

If you’re not involved with continuous integration and continuous deployment/delivery (CI/CD) or other software development automation, you might not be familiar with the name Codecov.

Codecov provides a hosted service that tells developers how much of their source code has been checked by the automated tests that are part of their software build process. It’s a metric called code coverage. Complicated software projects can have thousands of source code files. Knowing the code coverage tells you how effective your testing actually is because untested code might harbor bugs.

There are many possible paths of execution inside non-trivial code. Writing tests that check all of the paths is difficult. Basically, you’re writing code to find flaws in other code. But if your test code isn’t designed and written correctly it won’t cover all possible execution paths. That means there’ll be holes in your code coverage.

Codecov reads the output generated by your build process and produces reports that show exactly which execution paths your test code has missed. You can add tests to cover those regions or modify the logic of existing tests to make them thoroughly exercise the routine they’re supposed to test.

Code coverage is vital to producing stable software, especially with large teams of developers. You can tell how seriously this is taken by looking at some of Codecov’s 29,000 customers. Organizations like Kubernetes, Mozilla, HashiCorp, and Atlassian all rely on Codecov for their code coverage reports. And because Codecov is free to open-source projects, there are thousands of open-source projects using Codecov too.

That’s why it is such a good target for cybercriminals.

Supply-Chain Attacks

A supply-chain attack compromises a supplier or service provider in order to compromise the real targets—all of their customers and users. If you want to poison an entire town you’re not going to go house to house. You’ll poison the water treatment plant and wait.

This is what happened in the recent SolarWinds attack. SolarWinds wasn’t the target, they were just an efficient route for the threat actors to get at all of SolarWinds’ customers. And because Solarwinds’ customers for the most part were managed service providers and outsourced IT support organizations, all of their customers were exposed to risk as well.

RELATED: SolarWinds Hack: What Happened and How To Protect Yourself

The Codecov Attack

On April 15, 2021, Codecov publicly disclosed that a Bash script used to upload files to its servers had been modified by threat actors. The initial compromise at Codecov is thought to have occurred in late January 2021. The compromised script collected sensitive information from customers’ continuous integration environments and uploaded that information to the attacker’s server. Access credentials such as ID tokens and API keys as well as anything stored in environment variables were harvested by the modified script.

It was a classic supply-chain attack. By injecting a single line of code into Codecov’s Bash uploader script, the threat actors had the means to access the continuous integration environments of all customers that used the script. And because that script is used in three of Codecov’s uploading routines—Codecov-actions, CircleCl Orb, and Bitrise Step—very few Codecov customers would not be exposed to this risk.

That doesn’t mean all customers have been breached, just that they have been exposed to the threat of unauthorized access. HashiCorp is one customer known to have been compromised, while others such as Atlassian and Hewlett Packard Enterprises have not discovered any evidence of compromise.

The threat actors gained access to the Codecov infrastructure through a poorly configured Docker container. They added a line to the Bash uploader script at line 525 in the file. We can isolate the new line using the  colordiff command. The diff command would work just as well, but the colordiff output is a little easier to read, being color-coded. It’s available in all Linux distribution repositories.

The IP address of the attacker’s server was 104.248.94.23, which is a virtual server hosted by Digital Ocean.

Needless to say, the cybercriminals won’t use an IP address that points back to themselves.

As soon as Codecov discovered the compromise they took steps to close down the unauthorized access, communicate to their customers, and investigate the incident. Codecov has:

Collaborated with Digital Ocean to have the threat actor’s server taken down. Regenerated and updated all affected or potentially affected Codecov credentials and closed off the unauthorized access that allowed the uploader script to be modified. Checked their infrastructure logs to determine how the unauthorized access was possible, and which authentication key was used. Improved or implemented network monitoring and auditing tools at key points in their infrastructure to detect and prevent a recurrence of this type of attack.

Affected? What You Should Do Now

Codecov has emailed all customers they believe are at risk. Whether you have heard from them or not, if your organization is a customer of Codecov you have to assume that you are at risk of compromise.

Because credentials, connection keys, and other sensitive and confidential information are passed from step to step of the CI/CD process, the threat actors may have harvested those details. They may have been able to gather and retrieve:

Any authentication or privilege credentials, tokens, or keys that were accessible to the script while customer CI/CD processes were running. Any third-party services, data, or source code that could be accessed by the CI/Cd processes. The git remote repository details.

You can easily check what is in the environment variables of your CI/CD environment by using the  env command and piping it into less to make it more manageable:

Review the output and if there is anything that is sensitive or permits any kind of access, change the credentials on that account, platform, or service. If you are using a local copy of the Bash uploader script you are unlikely to have been affected, but you’re still encouraged to replace that local script with the latest version. Audit your system for attempted use by the invalidated credentials and keys. If attempts are detected, it means the threat actors are trying to use the information they’ve exfiltrated from your CI/CD platform to get in.

Act defensively. If there is any suspicion or possibility of compromise, regenerate the relevant credentials and keys immediately.