how to reduce memory footprint of a rails app

rubyonrails-talk
1

Hi-

My app has reached the memory limit allotted by my host. Can someone tell me how I go about reducing the memory footprint? Does rails load every single gem it finds, and can I somehow throw away the ones I don't need? Any other advice or suggestions for strategies or tools to diagnose where memory is being used would be appreciated.

Thanks, Dino

2

Are you running many mongrels?

If you are, a solution that is going to use less memory is running on JRuby with rails thread-safety enabled.

3

dino d. wrote:

My app has reached the memory limit allotted by my host. Can someone tell me how I go about reducing the memory footprint? Does rails load every single gem it finds, and can I somehow throw away the ones I don't need? Any other advice or suggestions for strategies or tools to diagnose where memory is being used would be appreciated.

The real question is: what's hogging up memory? Use theory of constraints: find the parts of your application that use the most memory, optimize it, and move on to the next battle.

You should profile and benchmark your application to find out. There's lots of tools you could use for that, including railsbench and the NewRelic/FineRuns/Scout facilities.

Depending on the time you have on your hands it might be cheaper to up the memory. RAM's cheap and developer time typically isn't.

4

thanks for the replies. i'm on dreamhost running passenger. their support is horrible, it takes 48 hours to get an e-mail response. anyway, i'm just looking to patch up my memory use in the app for the time being, and i was wondering how i go about decreasing the memory footprint by either optimizing code or removing unnecessary or unused gems or plugins. so the two generic questions are :

1) how do you/can you detect and removed unused gems or modules? 2) what kinds of things make memory bloat? can you give me a push in the right direction based on experience? do i start in my controllers or models, etc.?

i have no experience whatsoever in assessing where memory is used in a rails app and would appreciate any common patterns of opportunity for mem reduction.

thanks a lot. dino

5

dino d. wrote:

1) how do you/can you detect and removed unused gems or modules?

Ideally you've never added anything you don't need and removed things that became deprecated.

If not: do you have a test suite? Tracking code coverage or just removing things to see if tests break is a sure-fire way of knowing what's required and what's not.

2) what kinds of things make memory bloat? can you give me a push in the right direction based on experience? do i start in my controllers or models, etc.?

In no particular order:

- Having many routes - Using gems that leak (hpricot, feed-normalizer, old versions of RMagick) - Calling AR#size instead of AR#count where applicable - Retrieving large sets of data or columns where you don't need them - Loading binary files into memory

Please do add to the list.

6

Hi Dino,

Any chance you are running your development environment instead of production environment?

If you're running your dev env, and it's config'd like a typical default dev env, each request to the app will re-load the entire stack, and that stack will be cached. This is so that any changes you make to the app are immediately reflected in the app on the next request made on the app, which is what you'd want when your dev'ing your app. However, the longer you run under dev env, the more memory that dev env process will consume. This is not the case with a typical production env setup, in which app code mods would require an app restart to pick up such mods.

Regardless, if you want to log your app instance(s)'s resource usage upon each request for analysis purposes (and you're running under linux/unix) you could do something like:

  # in app/controller/application.rb:   ...   before_filter :log_resource_usage

  ...   private

  def log_resource_usage     # see ps man page: http://unixhelp.ed.ac.uk/CGI/man-cgi?ps     x,psr,etime,pcpu,pmem,rss,vsz = `ps -o psr,etime,pcpu,pmem,rss,vsz -p #{Process.pid}`.split('\n')[1].split(/\s+/)     logger.info("***INFO: resource_usage: rails_env=#{RAILS_ENV} pid=# {Process.pid} psr=#{psr} etime=#{etime} pcpu=#{pcpu} pmem=#{pmem} rss=# {rss} vsz=#{vsz} req_method=#{request.method} req_uri=#{request.env ['REQUEST_URI']}")   end

  ...

which will produce log entries like:

***INFO: resource_usage: rails_env=development pid=22401 psr=0 etime=14:38:23 pcpu=0.0 pmem=3.1 rss=32620 vsz=59952 req_method=get req_uri=/testapp/some/such/request

Jeff