The bootstrap-sass RubyGems Supply Chain Attack

What happened

On 26 March 2019, an attacker published bootstrap-sass 3.2.0.3 to RubyGems with a hidden backdoor. The attacker also removed the clean 3.2.0.2 release before the upload, so that new installs of the 3.2.0.x line picked up the bad version. Snyk and LWN both report this.

The backdoor lives in a new file, lib/active-controller/middleware.rb. The code changes a Rails method so that it reads an HTTP cookie named ___cfduid, decodes the value with base64, and runs the result with eval. The cookie name has three underscores. It copies the name of the real Cloudflare cookie, __cfduid, which has two. The code runs only when the Rails app is in production mode. Anyone who can send a request to such an app can then run commands on the server.

The code did not appear in the public GitHub repository. Security researcher Derek Barnes found the difference between the published gem and the source and reported it on the day of release. The malicious version left RubyGems less than one hour after the report, according to Snyk's timeline. The maintainers published 3.2.0.4 on 3 April 2019. It has the same content as the clean 3.2.0.2.

This case is a gem that gained a capability it never had. A styling library has no reason to read cookies or call eval. Vigilance compares the version you trust with the new version and reports the new file and the new power.

Affected versions

Only bootstrap-sass 3.2.0.3 is affected.

• 3.2.0.3: backdoored, published 26 March 2019.
• 3.2.0.2: the last clean release before the attack, from September 2014 per LWN.
• 3.2.0.4: clean re-release of 3.2.0.2, published 3 April 2019.
• 3.4.x: not affected.

LWN reports 1,477 downloads of 3.2.0.3. Snyk estimates about 1,670 repositories that use the gem directly. The app is at risk only if it ran 3.2.0.3 in production.

Indicators of compromise

These indicators come from the Snyk and LWN reports.

• A gem folder named bootstrap-sass-3.2.0.3.
• The file lib/active-controller/middleware.rb inside the gem.
• Requests with a cookie named ___cfduid (three underscores) in web server or proxy logs.
• SHA256 of the 3.2.0.3 gem, per Snyk: 366d6162fe36fc81dadc114558b43c6c8890c8bcc7e90e2949ae6344d0785dc0

How to check

Look for version 3.2.0.3 in your lock file and in your installed gems. Then search your logs for the three-underscore cookie.

grep -n "bootstrap-sass" Gemfile.lock
gem list bootstrap-sass
grep -rF "___cfduid" /var/log/

The grep -F match on three underscores does not match the real __cfduid cookie. Check every environment that built from the lock file, including CI caches and container images.

What to do now

1. Update to 3.2.0.4 or a later release. Run bundle update bootstrap-sass and commit the new Gemfile.lock.
2. Redeploy every production host that ran 3.2.0.3.
3. Search logs from 26 March 2019 onward for the ___cfduid cookie. A match means someone sent code to the app.
4. If a match exists, treat the server as compromised. Rotate the secrets that the app can read, such as database passwords, API keys and session secrets.
5. Pin gem versions in the lock file and review each new gem version before you deploy it.

What Vigilance showed

Vigilance compares the version you trust with the new one and names the file that gained a new capability. The block below is rebuilt from the public reports in the words Vigilance prints. It is not a captured scan, because the malicious release is not redistributed.

files scanned: 89 (1 Added)

HEADS UP  1 file can now do things the old version could not. The rest changed and gained nothing.

NEW FILE   lib/active-controller/middleware.rb
           It downloads from the internet and runs a hidden, encoded command.

Sources

1. snyk.io/blog/malicious-remote-code-execution-backdoor-discovered-in-the-popular-bootstrap-sass-ruby-gem/
2. lwn.net/Articles/785386/