5 Ways to Manage Environment Variables with direnv

If you find yourself frequently managing environment variables, especially project-specific variables, direnv can be a huge time saver.

Essentially, you define directory-specific variables in a .envrc file (or optionally .env), and those variables will be set when you enter that directory — or one of its children — and unset when you leave. The file is treated as a full-blown shell script, so you can add complicated logic to determine how values are set.

The installation process is well documented on the official site, so we won’t cover it here, but once you have things set up, here are a few examples of how you might use direnv.

1. Application Settings

Many applications expect to get certain information from your environment, like the DJANGO_SETTINGS_MODULE variable for running Django commands. You can set this and other optional variables automatically by adding something like this to .envrc:

export DJANGO_SETTINGS_MODULE=settings.testing
export DATABASE_HOST=db.local
export DATABASE_PORT=5432
export DATABASE_USER=admin
export DATABASE_PASSWORD=secret

2. Secrets

You should avoid storing secrets unencrypted on disk unless they only apply to services any local user would have access to anyway. There are several ways to do this, like running a Vault or 1Password command to retrieve the secret:

export API_KEY=$( op item get AWS --field API_KEY )

To build on our previous 1Password post, you can refer to your secrets using op:// URIs instead:

export API_KEY="op://Private/hdlzmqagfnf5bk5yxdgmyvamvq/API Access/Access Key ID"
export API_SECRET="op://Private/hdlzmqagfnf5bk5yxdgmyvamvq/API Access/Secret Access Key"

Then use op run -- your_app to expose the actual values at runtime.

3. Compiler Options

In many cases, building packages from source will require additional environment variables. A common scenario on macOS is referring to libraries installed using Homebrew. For example, if running pip install or poetry install in your project depends on OpenSSL 1.1, you can have these variables ready and waiting automatically:

export LDFLAGS="-L/opt/homebrew/opt/openssl@1.1/lib"
export CPPFLAGS="-I/opt/homebrew/opt/openssl@1.1/include"

4. Python Virtual Environments

You can use direnv to automatically activate (and even create) a virtual environment when you enter a project’s directory. I usually prefer to be explicit about which version of Python is currently active — and I might want to keep a virtualenv active even when leaving the project’s directory — so I don’t personally use this very often, but it’s an option you should be aware of.

You could do this in a straightforward way:

export VIRTUAL_ENV=$PWD/.venv
PATH_add .venv/bin

But with a bit of configuration, you can have a much more sophisticated process using direnv’s layouts. Add this to ~/.config/direnv/direnvrc:

realpath() {
    [[ $1 = /* ]] && echo "$1" || echo "$PWD/${1#./}"
}

layout_python-venv() {
    local python=${1:-python3}
    [[ $# -gt 0 ]] && shift
    unset PYTHONHOME
    if [[ -n $VIRTUAL_ENV ]]; then
        VIRTUAL_ENV=$(realpath "${VIRTUAL_ENV}")
    else
        local python_version
        python_version=$("$python" -c "import platform; print(platform.python_version())")
        if [[ -z $python_version ]]; then
            log_error "Could not detect Python version"
            return 1
        fi
        VIRTUAL_ENV=$PWD/.venv
    fi
    export VIRTUAL_ENV
    if [[ ! -d $VIRTUAL_ENV ]]; then
        log_status "no venv found; creating $VIRTUAL_ENV"
        "$python" -m venv "$VIRTUAL_ENV"
    fi

    PATH="${VIRTUAL_ENV}/bin:${PATH}"
    export PATH
}

And then you can do this in your .envrc:

export VIRTUAL_ENV=venv
layout python-venv

There are several advantages to this approach:

You can refer to the virtualenv folder using a relative path. (If you don’t define a path, $PWD/.venv will be used by default.)
If the virtualenv doesn’t exist, it will be created and activated automatically.
You can define which Python executable to use with something like layout python-venv python3.9.

There are built-in layouts for Python and other languages that ship with direnv. See the wiki for details on layouts and other useful features in the stdlib.

5. Inheritance

As a final tip, you might have a project with a few variables common to several applications, and some that are application-specific. For example, you might use the same AWS credentials for all of a client’s apps, but different databases. To handle this case, you can create .envrc files at multiple levels.

In ~/Projects/Client/.envrc:

export API_KEY="op://Private/hdlzmqagfnf5bk5yxdgmyvamvq/API Access/Access Key ID"
export API_SECRET="op://Private/hdlzmqagfnf5bk5yxdgmyvamvq/API Access/Secret Access Key"

In ~/Projects/Client/some_app/.envrc:

source_up
export DATABASE_URI=postgresql://user:pass@localhost:6432/some_app

The source_up is the important bit here. It tells direnv to search for and use other .envrc files in one or more parent directories.

Conclusion

Take a look at the direnv site to familiarize yourself with the security model and spark other ideas for streamlining your workflow. You won’t regret it.

Blog