Untangled Development

Minimal Environment Variables Setup for Django

Joining a new team? I rarely have not run into this problem when doing so: I try to run a project locally and it doesn’t work on my machine.

Onboarding others to your team? They try to run the project locally and it doesn’t work on their machine.

The issue seems to always boil down to environment issues. This post tries to go through some “steps”, or “best practices” if you will, on how to avoid running into these.

Env variables should not be in source code

Yes. Sure. And yet, for the project to run without issues, all team members should know which environment variables should be set.

One way to do this would be to document the expected environment variables in the project’s README.md or similar.

A better way would be to expect the environment variables to be in a file such as .env or .envrc in the project’s home directory. Load the contents of this file by running:

source .env

where .env contents would be:

export API_KEY=SECRET
export API_ENDPOINT=www.secret.com

While the contents of the .env or .envrc files would be:

  • specific for environment/user
  • excluded from source control

you should store a copy with dummy values in a corresponding .env.example or .envrc.example file.

This approach:

  • Allows storing the expected keys in a more intuitive way. This is easier to (remember to) maintain compared to a README.md file.
  • Allows the developer to cp .env.example .env and replace the dummy keys to get going.

This file shows all Django project environment variables the project uses. Team mates (or your future self) can refer to .envrc.example file going forward.

If running the project directly on your dev machine, there’s one more step. Have the environment variables in .env or .envrc loaded automatically whenever you access that directory. To achieve this I’ve been using direnv on Mac for years and it does its job well.

Or add a statement in the target environment to run source .env. To conclude, while environment variables should not be in source code:

  • document environment variables somewhere that can be shared with your team (or your future self)
  • document them in a way that is easy to manage/reuse
  • where possible have the environment variables loaded automatically

Example: the database connection string

A project setting which is usually different across environments is the database connection. In Django this is set using the DATABASES setting, docs here.

One way to manage this is by commenting out the default local, staging and prod keys. Instead, configure the value for the database connection as an environment variable.

For example, if you’re using dj-database-url to set up the DATABASES setting, a line in your .envrc would be:

export DATABASE_URL="postgres://postgres:secret@db:5432/testdb"

wherein the connection string for Postgres is of this format:

postgresql://user:password@host:port/database_name

Your settings file will then need to set DATABASES this way:

DATABASES = {"default": dj_database_url.config(conn_max_age=600)}

Update 2020-06-03: python-dotenv & django-environ

Including python-dotenv and django-environ for completeness’ sake.

python-dotenv

A Reddit user commented:

A helpful library for avoiding the need to manually source .env when you run the project (and also figuring out how to do that for your executing process in production) is Python DotEnv. Having a sample .env file is important, and you’ll still need to document what needs to happen with the .env file to get things running, but it removes the extra manual steps whenever you want to execute the codebase.

As the Reddit user commented, python-dotenv removes the need to have the .envrc (or .env) loaded in your environment by adding this code to the settings.py module:

from dotenv import load_dotenv
load_dotenv()

With this approach the settings file is loading the environment variables. In the previous approach the host OS environment was doing this.

So this approach is helpful in this way:

  • Enforces all team members to have environment variables in their .env file.
  • Expects you to define environment variables in .env files across local, staging, production, etc. Unless you want to riddle your settings file code with if statements.

Depending on how you look it at, the “advantages” above might be “disadvantages”. I.e. your Django code is responsible of loading “host” environment variables. But again, this might be an advantage as team members know exactly what to expect and where.

Same suggestions with keeping .env.example (or .envrc.example) up-to-date apply.

django-environ

I discovered this package following a reply to my tweet about this post. Its philosophy is like python-dotenv. My take is it makes the process more Django-specific. Again, whether this is a good or bad thing, depends on how you look at it.

I’ll quote its docs directly:

The idea of this package is to unify a lot of packages that make the same stuff: Take a string from os.environ, parse and cast it to some of useful python typed variables.

As with any Django-specific package, check the latest version supported by the package. And whether this works for you.

Conclusion

Aim to separate environment-dependent parts from the ones which are env-independent. Based on experience, the earlier you start this separation, the better. This keeps the tendency to adopt “local config exceptions” to a minimum. And instead move towards “12 factor app” concepts.

This pays off. Pinky promise.

Comments !