I'm involved in upgrading a large Rails app from 2.3.4 to 3.0.7. We
noticed the Rails 3 version of the app uses much more memory (causing
monit to continually restart our mongrels). Using REE 1.8.7.
After many experiments and lots of acrobatics with memprof (described
in detail at http://railski.blogspot.com/2011/05/rails3-leaks-more-memory.html
with code and heap dumps at https://github.com/pkmiec/leakage), I
believe the problem is related to lazy initialization.
Rails 3 defers lots of initialization until later in order to speed up
start up times. While handling requests, Rails initializes parts of
the app needed to fulfill the request. For example, a model is
required during start up (assuming it's in the eager load path), but
its columns are not known and attribute methods not created until they
are needed (for example when an instance of the model is created).
What I observed (and confirmed with memprof) is that controller
instances (and everything related to them) stick around on the heap if
some lazy initialization occurred during the request. So far I've
identified attribute methods (in particular define_attribute_methods),
controller's view_context_class, and controller's config to cause
leaks. What these things have in common is scopes that are assigned to
global or class level variables and eigen / singleton class madness.
I can force the initialization to occur in an after initialize hook
(call define_attribute_methods and protected_attributes on every
model, call view_context_class and config on every controller) and
prevent any controller instances from hanging around the heap. This
tricks works in the simplest Rails 3 app as well as in the large app
that's going from 2.3.4 to 3.0.7.
Looking at the heap data, I can't find a relationship between the
"things" involved in lazy initialization and "things" related to the
controller instance other than they share the same thread that created
them. This leads me to believe the problem I'm seeing is really a ruby
vm problem. I've tried different version of ruby 1.8.7 and they all
exhibit the same behavior.
Am I missing something obvious? Can someone explain why pre-loading
causes no controller instances to be left hanging on the heap (and
therefore no memory issues with the large app)?
I'd like to repeat my experiments with Ruby 1.9, but memprof doesn't
currently support 1.9. Is there another approach to inspect the heap