Doing code releases with ZFS

Plone backups can be painful

Have you ever wanted to release a new version of your site while maintaining a way to quickly roll back if something goes wrong? ZODB sizes are always growing, so how do you backup a state in time of the blob storage that can be even bigger? You could use the collective.recipe.backup in your buildout and take an incremental backup right before the release. However, this could still take time to create, and even more time to use as your backup rollback plan.

Filesystem snapshots are easier

At Six Feet Up, we love deploying onto FreeBSD. One of the really nice features of the OS that we can leverage is the ZFS filesystem to make our snapshots.

What is ZFS?

ZFS is the last filesystem you will ever need. I'll let you jump over here to read the details about it, but here are the basics:

• Sun developed ZFS
• Oracle bought them
• Oracle close-sourced the project
• The community spawned OpenZFS

OpenZFS is an awesome filesystem that has features beyond that of a standard system. It is a volume manager among other things, though the bit we care about is making snapshots and quickly rolling back if needed.

Using ZFS snapshots with Plone

Doing a release with ZFS ensures confidence that you can roll back in a matter of seconds. Here is how it works.

We will have our buildout on a ZFS dataset called data/buildout. Just before we checkout our tag for the latest release, we do the following:

zfs snapshot data/buildout@pre-release

You'll notice this happens instantaneously. ZFS does all of this on the block level and keeps track of the data changes as we go forward from this point.

One note:  If you don't clean up your snapshots later, they will continue to grow and take up space as changes happen to the file system.

If all goes well, you can just remove the snapshot like this:

zfs destroy data/buildout@pre-release

If all went bad, no problem, just stop all running processes that would have files open on that dataset and do this:

zfs rollback data/buildout@pre-release

Instantly, you will be transported back to that point in time and can restart your services as if nothing ever happened.

Day Saved!

As a developer, you can also use this technique on your local system. For example, you can use it to test and rollback migration steps. I've been applying this technique on OS X using MacZFS to create ZFS filesystems from plain files as the backend. I then symlink them into place and test and rollback til' my heart's content.

Interested in this topic? Want to learn more about ZFS? Leave a note in the comments and be sure to sign up for our Plone & Python How-To digests to receive more how-to guides as soon as they are published!

1. Fujitsu, "Solaris ZFS of Solaris™ 10 Operating System function," [Image]  http://www.fujitsu.com/global/products/computing/servers/unix/sparc-enterprise/software/solaris10/zfs/