I18n::Backend::KeyValue - available_locales, translations methods very slow

rubyonrails-talk
1

My team has been experimenting with I18n::Backend::KeyValue as an alternative to the Simple backend that comes packaged by default with Rails. We have been trying out using different key-value stores, such as RocksDB and BerkeleyDB. We have two main goals: 1) reduce Ruby process memory by storing translated strings externally, rather than in memory as the Simple backend does; and 2) improve startup time by pre-seeding the key-value store with the translated strings, rather than reading/parsing the YAML files at startup as the Simple backend does.

This seemed like a great idea, but in practice we’ve hit some snags. Mainly, the available_locales method and the translations method of the KeyValue store are both very slow.

I noticed that documentation for KeyValue does state that available_locales is slow, and you should avoid calling it on a KeyValue backend. However, since we are planning to use KeyValue as the primary backend, in our case it doesn’t seem reasonable to simply avoid calling this method. We would probably need to call it eventually.

I also noticed that the implementation of KeyValue#translations basically reads the contents of the key-value store into a Hash. This seems to mean that we wouldn’t actually get any memory savings by using KeyValue, because eventually at some point the translations would be read into memory anyway.

Does anyone have experience using KeyValue as their primary I18n backend?

2

The method is indeed very slow, as it maps the whole key set to each key prefix. So it has to load the whole list of keys into memory (at every method call). You can and should memoize it yourself somewhere (maybe in a class attribute?)

The memory usage is instead much better (unless you call the protected translations method, which forces the whole set of translations to be loaded). Single translation lookups do not consume more memory than needed because they are not cached, they will be garbage collected after being used.