I'm involved in upgrading a large Rails app from 2.3.4 to 3.0.7. We noticed the Rails 3 version of the app uses much more memory (causing monit to continually restart our mongrels). Using REE 1.8.7.
After many experiments and lots of acrobatics with memprof (described in detail at Railski: Rails3 leaks (more) memory? with code and heap dumps at GitHub - pkmiec/leakage: sample application to demonstrate memory leak in rails3), I believe the problem is related to lazy initialization.
Rails 3 defers lots of initialization until later in order to speed up start up times. While handling requests, Rails initializes parts of the app needed to fulfill the request. For example, a model is required during start up (assuming it's in the eager load path), but its columns are not known and attribute methods not created until they are needed (for example when an instance of the model is created).
What I observed (and confirmed with memprof) is that controller instances (and everything related to them) stick around on the heap if some lazy initialization occurred during the request. So far I've identified attribute methods (in particular define_attribute_methods), controller's view_context_class, and controller's config to cause leaks. What these things have in common is scopes that are assigned to global or class level variables and eigen / singleton class madness.
I can force the initialization to occur in an after initialize hook (call define_attribute_methods and protected_attributes on every model, call view_context_class and config on every controller) and prevent any controller instances from hanging around the heap. This tricks works in the simplest Rails 3 app as well as in the large app that's going from 2.3.4 to 3.0.7.
Looking at the heap data, I can't find a relationship between the "things" involved in lazy initialization and "things" related to the controller instance other than they share the same thread that created them. This leads me to believe the problem I'm seeing is really a ruby vm problem. I've tried different version of ruby 1.8.7 and they all exhibit the same behavior.
Am I missing something obvious? Can someone explain why pre-loading causes no controller instances to be left hanging on the heap (and therefore no memory issues with the large app)?
I'd like to repeat my experiments with Ruby 1.9, but memprof doesn't currently support 1.9. Is there another approach to inspect the heap with 1.9?
I have same issue on two applications.
Each request eat 50-70 mb of memory. (on development and production
modes)