# \[Rails 6\] Bug in caching key generation when working with active record collections? **URL:** \ **Category:** rubyonrails-core\ **Created:** [October 24, 2019, 6:56pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719 "2019-10-24T18:56:28Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Marc\_Kohlbrugge](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Kohlbrugge](https://discuss.rubyonrails.org/u/Marc_Kohlbrugge)\ **Post date:** [October 24, 2019, 6:56pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/1 "2019-10-24T18:56:28Z")
I’m running into a problem when using fragment caching with active record collections. Rails 6 uses a combination of collection.cache\_key and collection.cache\_version to determine whether a fragment cache is fresh or not. However, there is a scenario where the collection might change while collection.cache\_key and collection.cache\_version stay the same. It happens when you use a limit on the collection and then destroy one of those records, as long as it’s not the most recent one. This way collection.cache\_key will stay the same, because the query does not change. And collection.cache\_version will stay the same as well, because the collection count will remain unchanged as does the maximum updated\_at timestamp of the most recent record. I’ve build a sample app for demonstration purposes: > **[GitHub - marckohlbrugge/rails-6-collection-caching-bug](https://github.com/marckohlbrugge/rails-6-collection-caching-bug)** > > Contribute to marckohlbrugge/rails-6-collection-caching-bug development by creating an account on GitHub. The readme describes a way to reproduce the issue via the website itself, or the Rails console. Would this be considered a bug or expected behavior? Are there any known work arounds? ---
**Author:** ![Richard\_Schneeman](https://avatars.discourse-cdn.com/v4/letter/r/ec9cab/32.png) [@Richard\_Schneeman](https://discuss.rubyonrails.org/u/Richard_Schneeman)\ **Post date:** [October 27, 2019, 1:55pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/2 "2019-10-27T13:55:00Z")
Sounds like a bug. Can you move into an issue? ---
**Author:** ![Marc\_Kohlbrugge1](https://avatars.discourse-cdn.com/v4/letter/m/97f17d/32.png) [@Marc\_Kohlbrugge1](https://discuss.rubyonrails.org/u/Marc_Kohlbrugge1)\ **Post date:** [October 27, 2019, 2:40pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/3 "2019-10-27T14:40:04Z")
👍 [Modified Active Record collections may return same cache\_version when using limit · Issue #37555 · rails/rails · GitHub](https://github.com/rails/rails/issues/37555) I did find a few similar issues, but they all got marked as stale [https://github.com/rails/rails/issues/34408](https://github.com/rails/rails/issues/34408) [https://github.com/rails/rails/issues/31996](https://github.com/rails/rails/issues/31996) [https://github.com/rails/rails/issues/34093](https://github.com/rails/rails/issues/34093) I’m surprised this doesn’t appear to be a bigger problem. Perhaps I’m not supposed to fragment cache my paginated results. Using collection caching (`render collection: @posts, cached: true`) might be performant enough. ---
**Author:** ![Aaron\_Lipman](https://sea2.discourse-cdn.com/rubyonrails/user_avatar/discuss.rubyonrails.org/aaron_lipman/32/776_2.png) [@Aaron\_Lipman](https://discuss.rubyonrails.org/u/Aaron_Lipman)\ **Post date:** [October 28, 2019, 5:22pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/4 "2019-10-28T17:22:24Z")
Hi Marc & Richard, I’d categorize this as a bug. When generating a cache\_version for a collection, one solution might be to include a hash of the IDs of all items in the collection. This way, the cache\_version will change should an item be removed. I believe modifying the compute\_cache\_version method defined in [activerecord/lib/active\_record/relation.rb](https://github.com/rails/rails/blob/master/activerecord/lib/active_record/relation.rb) is the way to go about this. Marc, please let me know if you intend to submit a pull request. While I’m not on the Rails team, I’d be happy to offer any assistance and lobby for implementing a fix. (If you’re not interested in submitting a PR, I’d be excited to pick this up myself.) ---
**Author:** ![Daniel5](https://sea2.discourse-cdn.com/rubyonrails/user_avatar/discuss.rubyonrails.org/daniel5/32/3246_2.png) [@Daniel5](https://discuss.rubyonrails.org/u/Daniel5)\ **Post date:** [October 28, 2019, 8:31pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/5 "2019-10-28T20:31:17Z")
The full collection could be millions of records. Fetching the ids to hash might not be an option either, unless they can be hashed on the database side. Thanks, Daniel Heath ---
**Author:** ![Aaron\_Lipman](https://sea2.discourse-cdn.com/rubyonrails/user_avatar/discuss.rubyonrails.org/aaron_lipman/32/776_2.png) [@Aaron\_Lipman](https://discuss.rubyonrails.org/u/Aaron_Lipman)\ **Post date:** [October 30, 2019, 7:17pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/6 "2019-10-30T19:17:06Z")
Thanks for the flag, Daniel. On my first read of the code, I didn’t catch that an aggregate query was used to determine the max updated\_at timestamp (without fetching individual records) if a relation wasn’t loaded. Figuring out a database-side hashing function that works with all the flavors of SQL supported by Rails may prove tricky. I’m going to consider this some more, but an alternative approach might be to update the documentation to acknowledge this issue and explain how to customize a model’s cache\_version method according to one’s needs. ---
**Author:** ![Daniel5](https://sea2.discourse-cdn.com/rubyonrails/user_avatar/discuss.rubyonrails.org/daniel5/32/3246_2.png) [@Daniel5](https://discuss.rubyonrails.org/u/Daniel5)\ **Post date:** [October 30, 2019, 10:19pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/7 "2019-10-30T22:19:19Z")
I think it’s worth considering implementing per database, as the major ones all have something that’ll work and the keys don’t need to be stable across different data store implementations. Thanks, Daniel Heath ---
**Author:** ![Aaron\_Lipman](https://sea2.discourse-cdn.com/rubyonrails/user_avatar/discuss.rubyonrails.org/aaron_lipman/32/776_2.png) [@Aaron\_Lipman](https://discuss.rubyonrails.org/u/Aaron_Lipman)\ **Post date:** [November 4, 2019, 3:21pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/8 "2019-11-04T15:21:42Z")
Been thinking on this a few days, here are a couple potential solutions I’ve been considering: Hash-based: Sum the MD5 hashes of the IDs and append the last 32 hexadecimal digits of the sum to the cache version. Alternatively, use a mathematical hashing function like [Fibonacci Hashing](https://probablydance.com/2018/06/16/fibonacci-hashing-the-optimization-that-the-world-forgot-or-a-better-alternative-to-integer-modulo/). (The second approach wouldn’t accommodate non-numeric ID columns, but may be easier to implement across various SQL flavors without any conditional logic.) Both approaches need to account for the various 64-bit limits found in MySQL and Postgres, e.g. MySQL’s CONV() function. Last ID based: It occurs to me that when a record within a collection is destroyed, either the size of the collection or the ID of its last item will change. If we can reliably get that last ID, we can use it in conjunction with collection size to generate more robust cache keys. The LAST\_VALUE sql function might provide this. However, MySQL didn’t add LAST\_VALUE until version 8 (2018), but I think we can emulate the same functionality via MySQL session variables. I’m leaning towards the “Last ID” path. I think it’s a little more elegant and probably more efficient than calculating a sum when it comes to really large collections. Starting work on a pull request, but welcome other ideas/directions/considerations. ---
**Author:** ![Daniel5](https://sea2.discourse-cdn.com/rubyonrails/user_avatar/discuss.rubyonrails.org/daniel5/32/3246_2.png) [@Daniel5](https://discuss.rubyonrails.org/u/Daniel5)\ **Post date:** [November 4, 2019, 10:58pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/9 "2019-11-04T22:58:03Z")
The last ID isn’t stable if you replace an item within the collection though, right (assuming it’s via a HABTM, so the associated record doesn’t get touched)? ---
**Author:** ![Aaron\_Lipman](https://sea2.discourse-cdn.com/rubyonrails/user_avatar/discuss.rubyonrails.org/aaron_lipman/32/776_2.png) [@Aaron\_Lipman](https://discuss.rubyonrails.org/u/Aaron_Lipman)\ **Post date:** [November 5, 2019, 2:15pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/10 "2019-11-05T14:15:50Z")
Again, good point. There are some workarounds ([https://evrim.io/invalidating-caches-when-using-many-to-many-associations-in-rails/](https://evrim.io/invalidating-caches-when-using-many-to-many-associations-in-rails/)), but the absence of a touch: true option on has\_many relations makes the last ID approach less optimal. Also, I’m noting that support for the LAST\_VERSION function didn’t get added to SQLite3 until version 3.25.0 (2018). As much as I liked the idea of the last ID approach, I’m not seeing a clean way to implement that doesn’t require Rails devs to update their development environment. I’m going to continue to pursue a hash-based approach. Thanks again for your insights, Daniel! ---
**Author:** ![Richard\_Schneeman](https://avatars.discourse-cdn.com/v4/letter/r/ec9cab/32.png) [@Richard\_Schneeman](https://discuss.rubyonrails.org/u/Richard_Schneeman)\ **Post date:** [November 5, 2019, 4:37pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/11 "2019-11-05T16:37:51Z")
Love this passion and energy. Can you all move the conversation over to the issue so that it’s easier for other people have a similar problem to find or chime in? ---
**Author:** ![Aaron\_Lipman](https://sea2.discourse-cdn.com/rubyonrails/user_avatar/discuss.rubyonrails.org/aaron_lipman/32/776_2.png) [@Aaron\_Lipman](https://discuss.rubyonrails.org/u/Aaron_Lipman)\ **Post date:** [November 25, 2019, 1:16pm UTC](https://discuss.rubyonrails.org/t/rails-6-bug-in-caching-key-generation-when-working-with-active-record-collections/73719/12 "2019-11-25T13:16:50Z")
Thanks for the reminder, Richard. Wanted to ping folks that I’ve submitted a PR adding first and last IDs to cache versions: [https://github.com/rails/rails/pull/37724](https://github.com/rails/rails/pull/37724) (My reasoning for using bounding IDs over a checksum is contained in the summary, but I wanted to restate my gratitude for others’ input.) I greatly appreciate further review!