10 Tips for Upgrading to Python 3

Python 2’s end of life is finally here. On January 1, 2020, Python 2 support officially ends, which means there will be no new updates, no community support, and (worst of all) no new features!

However, if you haven’t ported to Python 3, don’t panic just yet. We’ve compiled our top 10 tips to make your upgrade easier:

1. Have good test coverage

You’ll be able to quickly squash bugs in your code with tools such as Coverage.py. These systems analyze your code to determine what was executed versus what failed in order to help identify where your problem is.

With good test coverage, you’ll also be able to keep your time, scope, and costs under control. The last thing the billing team wants to see is your timesheet where you spent 20 hours doing an upgrade that could’ve taken 1 hour with better coverage. If you can show Management that you can keep systems up-to-date at minimal expense, you’ll be more likely to get opportunities to keep everything running on the latest, greatest software. So let’s just keep those giant jumps between upgrades to a minimum!

2. Don’t regress on your Python 3 support

A great tool we’ve found is Pylint, a tool that looks at your code for errors all while making sure your code is following the appropriate compatibility for Python 3 code.

You can use the --py3k flag to lint your code to receive warnings when your code begins to deviate so you can stop the issues in their tracks. Installation is super easy, and you can use the tools across multiple platforms - making it ideal for your whole team.

As a bonus, this keeps your code compliant with the pep8 Python style guide AND is an Open Source tool.

3. Find your blocker dependencies

Can I Use Python 3 is a script that looks over your dependencies and lets you know how many are slowing down/stopping you from porting to Python 3. It can also show you which projects have no blockers, allowing you to start porting those to Python 3 first.

You can integrate this with your tests (see #2 above):

def check(requirements_paths=[], metadata=[], projects=[]):
    """Return True if all of the specified dependencies have been ported to Python 3.

    The requirements_paths argument takes a sequence of file paths to
    requirements files. The 'metadata' argument takes a sequence of strings
    representing metadata. The 'projects' argument takes a sequence of project
    names.

    Any project that is not listed on PyPI will be considered ported.
    """

‍

You can then integrate it into your tests like so:

import unittest
import caniusepython3

class DependenciesOnPython3(unittest.TestCase):
  def test_dependencies(self):
    # Will begin to fail when dependencies are no longer blocking you
    # from using Python 3.
    self.assertFalse(caniusepython3.check(projects=['ipython']))

_{Source: pypi.org/project/caniusepython3/}

4. Pay attention when using text compared to binary data

In Python 3, text and binary data are distinct types that cannot blindly be mixed together. You will likely have to take the time and verify when you are using text compared to binary data, and this cannot be entirely automated. Python 3 bytes does not behave like the old str from Pythons past, so we recommend reviewing your binary sequences types before doing any kind of debugging. In Fluent Python by Luciano Ramalho, there is a highly detailed chapter on text vs. bytes that details the nuances of the clear distinctions.

# Binary to Text
binary_data = b'I am text.'
text = binary_data.decode('utf-8')
print(text)

binary_data = bytes([65, 66, 67])  # ASCII values for A, B, C
text = binary_data.decode('utf-8')
print(text)

# Text to Binary
message = "Hello"  # str
binary_message = message.encode('utf-8')
print(type(binary_message))  # bytes

# Python has many built in encodings for different languages,
# and even the Caeser cipher is built in
import codecs
cipher_text = codecs.encode(message, 'rot_13')
print(cipher_text)

_{Source: https://www.devdungeon.com/content/working-binary-data-python}

5. Use feature detection at import to stay future-compatible

You might run into the issue where your code needs to know what version of Python to run on. By using feature detection, you’ll be able to tell if the version of Python you’re running will support your project/need. When the newer versions of Python are inevitably released, you’ll be able to assume that future versions are more compatible with Python 3 than Python 2.

We recommend avoiding version detection and relying on feature detection to prevent issues of the system detecting the wrong version (and to avoid future headaches down the line).

Example:

try:
    from importlib import abc
except ImportError:
    from importlib2 import abc

6. When maintaining Python 2 and 3 compatible code, turn to the "Future"

The __future__ module serves a multitude of purposes, but the ones we’re interested in are:

• To avoid confusing existing tools that analyze import statements and expect to find the modules they’re importing.
• To ensure that future statements run under releases prior to 2.1 at least yield runtime exceptions (the import of __future__ will fail, because there was no module of that name prior to 2.1).
• To document when incompatible changes were introduced, and when they will be — or were — made mandatory. This is a form of executable documentation, and it can be inspected programmatically via importing __future__ and examining its contents.
  _{(via https://docs.python.org/3/library/__future__.html)}

Using this module, you’ll be slowly accustomed to changes that are incompatible with your versions of Python. These are giving your words/symbols new meaning, and will allow you to use newer features in your older version of Python. You’ll want this to help future-proof your code for continuous updates.

7. Use Futurize to update your code

Python 3 is not backwards compatible (intentionally), so you’ll want to employ a script like Futurize to help make your code compatible for both Python 2 and 3. This will make porting from 2 to 3 simple and clean - without too many major changes to your code. Automatic conversions like these can greatly aide in your upgrade process, but keep in mind some of the limitations you might run into. Make sure to take a look at their documentation before making the jump.

8. Update your setup.py file to denote Python 3 compatibility

The whole point of this migration is to eventually stop supporting Python 2, so we highly recommend updating your classifiers in setup.py to contain Programming Language :: Python :: 3. You’ll want to consider adding classifiers for each major/minor version of Python that you’re going to be supporting as well. This will allow your machine to only support versions of Python that you want to work on, greatly reducing the risk you’ll backtrack to Python 2 on a new project.

9. Do trigger an exception when comparing bytes to strings or bytes to an integer

This tip relates back to #5, but is important to mention again. You can use the -bb flag to have the Python 3 interpreter trigger an exception when you’re comparing bytes to strings or bytes to an int. By raising this flag, you’ll be easily able to find the mistake much easier than if you were separating your text/binary data handling or indexing on bytes.

10. Look at Python 2 to Python 3 case studies

You’re not alone in this migration, nor should you be! There are countless examples of other folks who have made the port from Python 2 to Python 3. We recommend taking time to review case studies and other similar projects to yours to see what issues, successes, and workarounds others have found to common problems.

Here are a few of our favorite examples:

• https://diveintopython3.net/case-study-porting-chardet-to-python-3.html
• https://hackaday.com/2018/08/15/stop-using-python-2-what-you-need-to-know-about-python-3/
• https://portingguide.readthedocs.io/en/latest/
• https://www.toptal.com/python/python-3-is-it-worth-the-switch
• https://medium.com/@boxed/moving-a-large-and-old-codebase-to-python3-33a5a13f8c99

Overwhelmed yet? We can help! At Six Feet Up, we’ve successfully completed numerous Python 3 upgrades for clients and are happy to help with your upgrade project.

Sources & Resources:

• To learn more about our Python 3 Upgrade Campaign, head over to sixfeetup.com/upgrade
• Top Reasons to Upgrade to Python 3: https://sixfeetup.com/blog/why-you-should-upgrade-to-python-3
• Porting Python 2 Code to Python 3 Docs via Python.org
• How To Port Python 2 Code to Python 3 via Digital Ocean
• Python 2.7 Release Schedule via Python.org
• Best Practices for Compatible Python 2 and 3 Code via PyBytes
• Switching From Python 2 To 3 via Trey Hunner