I've recently deployed an app that is having some serious memory
consumption issue. Dropping mongrels quite frequently when the
threshold is met.
Initially, we chalked this problem up to RMagick (I know, I know, but we
needed watermarking). We quickly optimized things so all images were
only processed once then neatly stored in their own folder. When an
image is requested, we confirm that it doesn't exist before processing
it. Been tested, and this is working as it should.
According to some memory profiling we've done, there are a couple of key
pages that are creating large amounts of strings that are not being
reclaimed.
Has anyone ever experienced issues similar to this?
I've recently deployed an app that is having some serious memory
consumption issue. Dropping mongrels quite frequently when the
threshold is met.
Initially, we chalked this problem up to RMagick (I know, I know, but we
needed watermarking). We quickly optimized things so all images were
only processed once then neatly stored in their own folder. When an
image is requested, we confirm that it doesn't exist before processing
it. Been tested, and this is working as it should.
According to some memory profiling we've done, there are a couple of key
pages that are creating large amounts of strings that are not being
reclaimed.
Has anyone ever experienced issues similar to this?
Cheers,
DJ
I too am presently using monit to manage this until I can track down the leak.
Our mongrel's high water mark is 280 mb, and each of the 11 restart a few times per day.
Rmagick is in place, but isolated, and likely not the source.
There's an article somewhere (lost in my booksmarks) that helps track down leaks. As I suspect my leaks are very specific, I have not appealed to the general group, but perhaps this thread will turn out something interesting.
I believe our mongrels memory threshold is in the same ballpark. Here
are a couple of articles we've been looking at, following along with
these processes and using the memory analysis tool in the second has
revealed that the strings not getting reclaimed are blank..
I believe our mongrels memory threshold is in the same ballpark. Here
are a couple of articles we've been looking at, following along with
these processes and using the memory analysis tool in the second has
revealed that the strings not getting reclaimed are blank..
We've also been using bleak_house:
Evan Weaver
yet still can't nail down exactly whats going on here.
Jodi, what version of Rails are you using for you project?
Thanks,
Dave
We're on 1.2.3, mysql 5, ubunutu 6.06 LTS, ruby 1.8.6 (i686).
I'm deploying a new server (throw money at the problem), but will be getting to the leaks next week.
We've had the leaks since day 1, and although we have 1.8million plus pages, controller actions are limited - so I should be able to localize the area quickly. I'll update this thread with anything learned.
My image uploading is low volume, so this works fine for me. I think
it would work well even for much higher volumes, but I haven't
benchmarked the difference between a command line invocation and
RMagick. It's not likely to leak though.
My image uploading is low volume, so this works fine for me. I think
it would work well even for much higher volumes, but I haven't
benchmarked the difference between a command line invocation and
RMagick. It's not likely to leak though.
If you track down the problem please post it.
I don't have any answers yet, a few clues, but have some tools to suggest:
(the above is from a single run of hitting a single url for 15 minutes. working out a test approach presently)
The above is after a 30 minute run - showing 5511 objects lost in memached (funny enough also written by Evan - I've asked him about the leak). The __ leaks are supposedly internal to ruby - boy, that's a load of lost objects -
other than tzinfo (which I've heard is a culprit), it seems to me that routing is generating a load. Anybody have any info on routing and leaks?
I'm at a bit of a loss when digging info ruby leaks/garbage collection. If anybody has a good ref I'd appreciate it.
We'll we tracked down our memory issues. We had some complex join
queries going on and it appears that the AR objects created from said
queries were not getting reclaimed.
So if you are still having issues, and have some complex find_by_sql or
find statements using a lot/nested includes. Try breaking those up into
two queries.. Vast improvement here after doing that.