Blog

Background

For the past 12 years or so, FreeBSD has been an excellent server OS for Six Feet Up.  It is well engineered, well documented, and works the way that we expect an operating system to work.  One of the great things about FreeBSD is the ports system.  For the uninitiated, the ports system (or, ports “tree”) is a large repository of patches for software packages and build instructions to make those software packages compile for FreeBSD (in addition to other meta-information about dependencies, etc).  What is really great about the ports system is that it lets you build software exactly as you need it for the server/task you are working on.  For example, if you want a lean server, with the minimal amount of installed software, you have the opportunity to decide exactly how software packages will be built (with what options/dependencies), so that there is nothing installed that you don’t specifically want to be there.  You get all this just by navigating some menus while building the software packages that you want.

While the ports system is awesome at certain things, it does have some drawbacks.  First and foremost is the time necessary to build all the software for the system.  Now, for the naysayers out there, sure there are packages built with every release of FreeBSD, but those packages are built with the default options (not mine).  We find that for our customers, custom builds for our environment end up being the better choice.  The second big issue, for us, is the time required to keep the system software up to date on an ongoing basis.

Enter Tinderbox (from stage left)

Tinderbox is an automated package building system for FreeBSD ports, developed originally by Joe Marcus Clarke. It compiles, once and for all, the various software that our servers use and keeps them up-to-date, thus saving a lot of time for our systems administrators.

Tinderbox uses FreeBSD jails as build environments, combined with ports trees, to build packages for the architecture(s) needed.  You can build packages with the options that you want, and have them ready to deploy.  The options you define for the various packages work exactly like the regular FreeBSD ports system.  We maintain all our options files in Git, so we can create branches for older ports trees that we need to keep the options information for.

Enter pkgng (from stage right)

Pkng (or "pkg", as it is officially called in the ports system) is the “next generation” package manager for FreeBSD.  Now that Tinderbox (in the tinderbox-devel port) has support for pkgng repositories, it is possible to build packages with tinderbox, which can easily be turned into Pkgng repositories (with a single command).  Pkgng will let us upgrade our installed packages via “pkg update” and “pkg upgrade”.  Spiffy.

Putting it all together (stage center)

Although we now have Tinderbox building packages, we have one more issue that we need to address:  the ports tree is alive.  Perhaps referring to it as a tree is not just a continuation of existing jargon.  It is constantly changing in any number of ways:  software versions, default software for certain things, default options, etc.  We need to have snapshots of the ports tree in time, and be able to rebuild packages in those ports trees (overlooked important option, perhaps) without having to upgrade all the packages installed on a system that is in production.  Tinderbox comes to the rescue.  Tinderbox lets us have completely separate ports trees for the same build environment, and keep the built packages separate.  So, whenever we’d like to make a new ports tree (e.g., important vulnerability in a package we use), we can make a snapshot of what is current, and build packages for that new ports tree.  All the while, we still have the old ports trees, meaning we can continue to use/modify the packages for them, and deploy them if we really need to.  Further, maintaining these old packages gives us a way to roll-back, should we have an issue with a new package set.

Now that we have Tinderbox building packages in an organized way, and Pkgng repos available with our custom packages, it’s time for one last piece of chocolate-y goodness:  meta-ports for our environment.  Using the FreeBSD ports system structure, we created some meta-ports, which allow us to install sets of packages in a single command.  Yes, there are other, perhaps easier, ways to do this, but we liked the idea of having meta-ports/packages available to install sets of packages based on the server’s role.  The meta-ports also have the advantage of simplifying the amount of work that we have to do with tinderbox.  We have only a few ports to add to a given build environment, and all the necessary packages for our systems will be built.

Caveat emptor

This system is still in development for us here at Six Feet Up.  We would caution anyone who wanted to attempt such a thing to do lots of testing before deploying it in a production environment. But if you are up to the challenge, we will be happy to trade tips and tricks with you: just email us at support@sixfeetup.com.