Caching and .includes()

I’ve noticed that in my applications I often have to choose between the efficiency of caching and the efficiency of the .includes() method, and I’m wondering if these two are mutually exclusive. Am I doing something wrong?

Here’s an example. Let’s say I’ve got a Posts model, and each Post has_many Authors.

Controller:

@recent = Posts.order(created_at: :desc).includes(:authors)

``

View:

<% cache @recent do %>
<% @recent.each do |post| %>
<%= post.authors.map{ |a| a.name }.joins(’’) + post.title %>

<% end %>
<% end %>

``

Okay, so my issue here is:

If the cache key is just based on the Posts, then the controller code will pull the Authors association for no reason (we don’t actually need it, because the author names are cached until the Post models are touched). This makes me think I should remove the .includes() call from my controller, since it will cause an unnecessary db query 99% of the time.

On the other hand, if I remove the .includes() call from my controller, then whenever the cache does expire, I have an n-queries situation, which also seems bad.

Is there any way to have the best of both worlds here?

A related question:

I’ve also found it helpful to use lower-level caching in my Post model to cache the “recent” query like so:

post.rb

self.recent do
Rails.cache.fetch('recent_posts’, expires_in: 1.day) do
self.order(created_at: :desc).to_a
end
end

``

Here I have a similar problem. If I don’t include the .to_a at the end, then I’m only caching the scope, which will still evaluate and seemingly (at least in development) trigger a db query on every pageview. If I do include the .to_a then I forfeit the ability to use .includes() in my individual controller actions or views.

Again, is there a way to have the best of both worlds here?

Thanks for any ideas and pointers people might have! Cheers,

Brian

Okay, so my issue here is:

If the cache key is just based on the Posts, then the controller code will pull the Authors association for no reason (we don’t actually need it, because the author names are cached until the Post models are touched). This makes me think I should remove the .includes() call from my controller, since it will cause an unnecessary db query 99% of the time.

On the other hand, if I remove the .includes() call from my controller, then whenever the cache does expire, I have an n-queries situation, which also seems bad.

Is there any way to have the best of both worlds here?

The :preload variant of includes can be run after the query has run -

ActiveRecord::Associations::Preloader.new.preload(@recent, :authors)

Will load that association on @recent (you can preload multiple or nested associations in the same way. You could do this once you’ve gone down the cache miss case.

A related question:

I’ve also found it helpful to use lower-level caching in my Post model to cache the “recent” query like so:

post.rb

self.recent do
Rails.cache.fetch('recent_posts’, expires_in: 1.day) do
self.order(created_at: :desc).to_a
end
end

``

Here I have a similar problem. If I don’t include the .to_a at the end, then I’m only caching the scope, which will still evaluate and seemingly (at least in development) trigger a db query on every pageview. If I do include the .to_a then I forfeit the ability to use .includes() in my individual controller actions or views.

Again, is there a way to have the best of both worlds here?

You can use the same trick here.

Fred

I’ve noticed that in my applications I often have to choose between the
efficiency of caching and the efficiency of the .includes() method, and I’m
wondering if these two are mutually exclusive. Am I doing something wrong?

Here’s an example. Let’s say I’ve got a Posts model, and each Post has_many
Authors.

Controller:
@recent = Posts.order(created_at: :desc).includes(:authors)

View:
<% cache @recent do %>
  <% @recent.each do |post| %>
    <%= post.authors.map{ |a| a.name }.joins('') + post.title %>
  <% end %>
<% end %>
Okay, so my issue here is:

If the cache key is just based on the Posts, then the controller code will
pull the Authors association for no reason (we don’t actually need it,
because the author names are cached until the Post models are touched).
This makes me think I should remove the .includes() call from my
controller, since it will cause an unnecessary db query 99% of the time.

On the other hand, if I remove the .includes() call from my controller,
then whenever the cache *does* expire, I have an *n*-queries situation,
which also seems bad.

Is there any way to have the best of both worlds here?

Try and come up with a better cache key. I'm not even sure what happens
when you use an AR scope as a cache key. Does it load all the records
into an array and generate some funky composite key? Loading all the
posts to use as a cache key is not very efficient.

Do the Post models belong to something? One option is to use that parent
option as a cache key. Have the posts touch the parent object when they
change.

A related question:

I’ve also found it helpful to use lower-level caching in my Post model to
cache the “recent” query like so:

post.rb
self.recent do
  Rails.cache.fetch('recent_posts’, expires_in: 1.day) do
    self.order(created_at: :desc).to_a
  end
end

Here I have a similar problem. If I don’t include the `.to_a` at the end,
then I’m only caching the *scope*, which will still evaluate and seemingly
(at least in development) trigger a db query on every pageview. If I *do* include
the `.to_a` then I forfeit the ability to use `.includes()` in my
individual controller actions or views.

Have you actually benchmarked that this cache is effective? My guess is
instantiating objects out of the cache won't be much faster than
instantiating them out of the database.