Hi all, I have two types of environment variables:
- Secrets, like
DISCOURSE_API_KEY, and - Environment-specific settings, like
DISCOURSE_BASE_URL
I’d like to accomplish a few things:
- Crash the app quickly if required environment variables aren’t set
Note that certain environment variables are only required in production. In development or test, they should either have default values (e.g. api keys), or don’t need to be specified (e.g. storage bucket credentials).
- Reduce the number of environment variables needed to be specified to run the application in development and test
I’m up to 23, and some of the environment-specific settings seem a bit silly to have to add to my CI, for example. Providing environment-specific defaults seems like a reasonable solution.
To complicate this a bit further, my production setup involves my main long-running
webprocess, and adb-migrateprocess that’s run after a deployment, and simply runs the commandrails db:migrateand exits. This runs inproduction, but ideally I wouldn’t have to provide all the unnecessary environment variables to it (e.g. api keys), both for convenience and the principle of least privilege. But maybe this one doesn’t really matter.
- Make it easier to maintain the evolving list of environment variables
Having ENV.fetch() calls scattered throughout the application, and having to keep the default values consistent, feels messy. My gut tells me a config file that reads them into a config property would be easier to maintain.
So I looked to Rails::Application.config_for to do just that, and wrote a file called endpoints.yml (couldn’t think of a better name) that looked roughly like this:
shared:
storage: # fail if any of these are missing, except in test
endpoint: <%= ENV.fetch('STORAGE_ENDPOINT') %>
bucket: <%= ENV.fetch('STORAGE_BUCKET') %>
region: <%= ENV.fetch('STORAGE_REGION') %>
access_key_id: <%= ENV.fetch('STORAGE_ACCESS_KEY_ID') %>
secret_access_key: <%= ENV.fetch('STORAGE_SECRET_ACCESS_KEY') %>
development: &development
discourse:
base_url:
internal: http://host.docker.internal:3000 # used for api calls from this app
external: https://forums.mysite.local # used to render links for users
api_key: <%= ENV.fetch('DISCOURSE_API_KEY', 'dev') %> # fallback value
legacy_app:
base_url:
internal: http://legacy
external: https://legacy.mysite.local
api_key: <%= ENV.fetch('LEGACY_APP_API_KEY', 'dev') %> # fallback value
test:
<<: *development
storage: # we use in-memory storage for file uploads in test
endpoint: ~
bucket: ~
region: ~
access_key_id: ~
secret_access_key: ~
production:
discourse:
base_url:
internal: https://forums.mysite.org
external: https://forums.mysite.org
api_key: <%= ENV.fetch('DISCOURSE_API_KEY') %> # fail if missing
legacy_app:
base_url:
internal: https://legacy.mysite.org
external: https://legacy.mysite.org
api_key: <%= ENV.fetch('LEGACY_APP_API_KEY') %> # fail if missing
This seemed like the ideal solution, as it would fail if required environment variables were missing in production, but provide fallback defaults in development and test. Unfortunately, it threw a KeyError in development and test as well, because config_for evaluates the ERB before parsing the YAML and selecting the environment-specific subset of properties. So the ENV.fetch() calls in the production section are evaluated even in development and test environments.
So I ended up splitting endpoints.yml into three files, endpoints/{environment}.yml, and calling the appropriate one from application.rb via:
endpoints_config_path = Rails.root.join("config", "endpoints", "#{Rails.env}.yml")
config.endpoints = config_for(endpoints_config_path)
This did the trick, but feels a bit hacky, and it still fails when my database migration worker runs (because that worker only has/needs DATABASE_URL). I thought I’d check to see if there’s another good way out there.
Thanks for reading!
