What's a good way to require environment variables?

Hi all, I have two types of environment variables:

  • Secrets, like DISCOURSE_API_KEY, and
  • Environment-specific settings, like DISCOURSE_BASE_URL

I’d like to accomplish a few things:

  1. Crash the app quickly if required environment variables aren’t set

Note that certain environment variables are only required in production. In development or test, they should either have default values (e.g. api keys), or don’t need to be specified (e.g. storage bucket credentials).

  1. Reduce the number of environment variables needed to be specified to run the application in development and test

I’m up to 23, and some of the environment-specific settings seem a bit silly to have to add to my CI, for example. Providing environment-specific defaults seems like a reasonable solution.

To complicate this a bit further, my production setup involves my main long-running web process, and a db-migrate process that’s run after a deployment, and simply runs the command rails db:migrate and exits. This runs in production, but ideally I wouldn’t have to provide all the unnecessary environment variables to it (e.g. api keys), both for convenience and the principle of least privilege. But maybe this one doesn’t really matter.

  1. Make it easier to maintain the evolving list of environment variables

Having ENV.fetch() calls scattered throughout the application, and having to keep the default values consistent, feels messy. My gut tells me a config file that reads them into a config property would be easier to maintain.

So I looked to Rails::Application.config_for to do just that, and wrote a file called endpoints.yml (couldn’t think of a better name) that looked roughly like this:

  storage: # fail if any of these are missing, except in test
    endpoint: <%= ENV.fetch('STORAGE_ENDPOINT') %>
    bucket: <%= ENV.fetch('STORAGE_BUCKET') %>
    region: <%= ENV.fetch('STORAGE_REGION') %>
    access_key_id: <%= ENV.fetch('STORAGE_ACCESS_KEY_ID') %>
    secret_access_key: <%= ENV.fetch('STORAGE_SECRET_ACCESS_KEY') %>

development: &development
      internal: http://host.docker.internal:3000 # used for api calls from this app
      external: https://forums.mysite.local # used to render links for users
    api_key: <%= ENV.fetch('DISCOURSE_API_KEY', 'dev') %> # fallback value
      internal: http://legacy
      external: https://legacy.mysite.local
    api_key: <%= ENV.fetch('LEGACY_APP_API_KEY', 'dev') %> # fallback value

  <<: *development
  storage: # we use in-memory storage for file uploads in test
    endpoint: ~
    bucket: ~
    region: ~
    access_key_id: ~
    secret_access_key: ~

      internal: https://forums.mysite.org
      external: https://forums.mysite.org
    api_key: <%= ENV.fetch('DISCOURSE_API_KEY') %> # fail if missing
      internal: https://legacy.mysite.org
      external: https://legacy.mysite.org
    api_key: <%= ENV.fetch('LEGACY_APP_API_KEY') %> # fail if missing

This seemed like the ideal solution, as it would fail if required environment variables were missing in production, but provide fallback defaults in development and test. Unfortunately, it threw a KeyError in development and test as well, because config_for evaluates the ERB before parsing the YAML and selecting the environment-specific subset of properties. So the ENV.fetch() calls in the production section are evaluated even in development and test environments.

So I ended up splitting endpoints.yml into three files, endpoints/{environment}.yml, and calling the appropriate one from application.rb via:

endpoints_config_path = Rails.root.join("config", "endpoints", "#{Rails.env}.yml")
config.endpoints = config_for(endpoints_config_path)

This did the trick, but feels a bit hacky, and it still fails when my database migration worker runs (because that worker only has/needs DATABASE_URL). I thought I’d check to see if there’s another good way out there.

Thanks for reading!

With Rails credentials you can always call bang on the configuration value which will raise an error if that’s set Securing Rails Applications — Ruby on Rails Guides

This doesn’t fully solve what you’re looking to do and I’m also curious how others are utilizing inheritance with credentials mostly for the local environment. Right now you have to opt fully into one environment and you can’t for instance stack configuration to combine some common defaults for the local environment and then override a few settings for your development environment. I used to do this with the dotenv gem and layer the configuration which worked nicely.