Caching and .includes()

rubyonrails-talk
1

I’ve noticed that in my applications I often have to choose between the efficiency of caching and the efficiency of the .includes() method, and I’m wondering if these two are mutually exclusive. Am I doing something wrong?

Here’s an example. Let’s say I’ve got a Posts model, and each Post has_many Authors.

Controller:

@recent = Posts.order(created_at: :desc).includes(:authors)

``

View:

<% cache @recent do %> <% @recent.each do |post| %> <%= post.authors.map{ |a| a.name }.joins(‘’) + post.title %>

<% end %> <% end %>

``

Okay, so my issue here is:

If the cache key is just based on the Posts, then the controller code will pull the Authors association for no reason (we don’t actually need it, because the author names are cached until the Post models are touched). This makes me think I should remove the .includes() call from my controller, since it will cause an unnecessary db query 99% of the time.

On the other hand, if I remove the .includes() call from my controller, then whenever the cache does expire, I have an n-queries situation, which also seems bad.

Is there any way to have the best of both worlds here?

A related question:

I’ve also found it helpful to use lower-level caching in my Post model to cache the “recent” query like so:

post.rb

self.recent do Rails.cache.fetch('recent_posts’, expires_in: 1.day) do self.order(created_at: :desc).to_a end end

``

Here I have a similar problem. If I don’t include the .to_a at the end, then I’m only caching the scope, which will still evaluate and seemingly (at least in development) trigger a db query on every pageview. If I do include the .to_a then I forfeit the ability to use .includes() in my individual controller actions or views.

Again, is there a way to have the best of both worlds here?

Thanks for any ideas and pointers people might have! Cheers,

Brian

2

Okay, so my issue here is:

If the cache key is just based on the Posts, then the controller code will pull the Authors association for no reason (we don’t actually need it, because the author names are cached until the Post models are touched). This makes me think I should remove the .includes() call from my controller, since it will cause an unnecessary db query 99% of the time.

On the other hand, if I remove the .includes() call from my controller, then whenever the cache does expire, I have an n-queries situation, which also seems bad.

Is there any way to have the best of both worlds here?

The :preload variant of includes can be run after the query has run -

ActiveRecord::Associations::Preloader.new.preload(@recent, :authors)

Will load that association on @recent (you can preload multiple or nested associations in the same way. You could do this once you’ve gone down the cache miss case.

A related question:

I’ve also found it helpful to use lower-level caching in my Post model to cache the “recent” query like so:

post.rb

self.recent do Rails.cache.fetch('recent_posts’, expires_in: 1.day) do self.order(created_at: :desc).to_a end end

``

Here I have a similar problem. If I don’t include the .to_a at the end, then I’m only caching the scope, which will still evaluate and seemingly (at least in development) trigger a db query on every pageview. If I do include the .to_a then I forfeit the ability to use .includes() in my individual controller actions or views.

Again, is there a way to have the best of both worlds here?

You can use the same trick here.

Fred

3

I’ve noticed that in my applications I often have to choose between the efficiency of caching and the efficiency of the .includes() method, and I’m wondering if these two are mutually exclusive. Am I doing something wrong?

Here’s an example. Let’s say I’ve got a Posts model, and each Post has_many Authors.

Controller: @recent = Posts.order(created_at: :desc).includes(:authors)

View: <% cache @recent do %>   <% @recent.each do |post| %>     <%= post.authors.map{ |a| a.name }.joins('') + post.title %>   <% end %> <% end %> Okay, so my issue here is:

If the cache key is just based on the Posts, then the controller code will pull the Authors association for no reason (we don’t actually need it, because the author names are cached until the Post models are touched). This makes me think I should remove the .includes() call from my controller, since it will cause an unnecessary db query 99% of the time.

On the other hand, if I remove the .includes() call from my controller, then whenever the cache *does* expire, I have an *n*-queries situation, which also seems bad.

Is there any way to have the best of both worlds here?

Try and come up with a better cache key. I'm not even sure what happens when you use an AR scope as a cache key. Does it load all the records into an array and generate some funky composite key? Loading all the posts to use as a cache key is not very efficient.

Do the Post models belong to something? One option is to use that parent option as a cache key. Have the posts touch the parent object when they change.

A related question:

I’ve also found it helpful to use lower-level caching in my Post model to cache the “recent” query like so:

post.rb self.recent do   Rails.cache.fetch('recent_posts’, expires_in: 1.day) do     self.order(created_at: :desc).to_a   end end

Here I have a similar problem. If I don’t include the `.to_a` at the end, then I’m only caching the *scope*, which will still evaluate and seemingly (at least in development) trigger a db query on every pageview. If I *do* include the `.to_a` then I forfeit the ability to use `.includes()` in my individual controller actions or views.

Have you actually benchmarked that this cache is effective? My guess is instantiating objects out of the cache won't be much faster than instantiating them out of the database.