Safety and pip-audit: Comparing Security Tools

With a 650% increase in next-gen cyberattacks against open source tools recorded in the last year, securing your software supply chain is more important than ever. To that end, there are a number of tools you can use to protect all the Python tools and packages on your development computer.

One of these tools is Safety. Produced by PyUp, Safety is a Python package you can install with pip that scans dependencies for security and licensing issues. The package uses Safety DB, which is the company’s public open source vulnerability database.

While Safety is a great tool, it’s always good to shop around and research security tools to figure out what works best for your development setup. Let’s look at the pros and cons of using Safety and compare it to another alternative: pip-audit.

A Closer Look at Safety

Safety checks installed dependencies for known security vulnerabilities. It uses Safety DB by default, but it can be upgraded to use PyUp’s paid service which offers advanced features including:

• a more up-to-date vulnerability database (Safety DB is only updated monthly);
• the ability to sync that database in a local system; and
• a Common Vulnerability Scoring System (CVSS), which is an industry standard for assessing vulnerabilities.

Safety works with Python 3.5 or higher. While Safety has stopped supporting Python 2.7, you can run Safety from a Python 3 environment to check the requirements file for your Python 2.7 project. Learn more about Safety here.

pip install safety
safety check -r <package file 1> -r <package file 2>

What’s good about Safety?

• It is easy to install, integrate with CI and manage;
• it has a user friendly output for either table or JSON (see image below);
• Docker image is available; and
• full reports are available.

What could Safety improve?

• The open source Safety DB is only updated monthly;
• the paid versions are pricey; and
• the output does not display the severity of the vulnerability, just the ID.

Pip-audit as an Alternative

Pip-audit is a tool for scanning Python environments for packages with known vulnerabilities. It uses the Python Packaging Advisory Database via the PyPi JSON API as a source of vulnerability reports.

The tool was developed by Trail of Bits with support from Google. It is licensed under Apache 2.0 and is free to use. You can learn more about pip-audit here.

pip install pip-audit
pip-audit -r <package file 1> -r <package file 2>

What’s good about pip-audit?

• It is free under the Apache 2.0 license;
• it has a user friendly output for either table or JSON; and
• it is easy to implement and maintain.

What could pip-audit improve?

• It lacks good documentation; and
• it is slower than Safety.

Our Recommendation

Safety and pip-audit are similar solutions that do the same job. However, because Safety has a paid version that you can upgrade to if needed — and good documentation — I would recommend Safety over pip-audit for people wanting to protect the Python tools and packages on their development computers.

For a great explanation of the software supply chain, watch PyUp CEO, Justin Womersley, share security best practices during his February 2022 presentation at IndyPy.