I have a Rails App that is hitting a fairly big database (several
million rows of data).
The app runs really well as I have enough ram on the servers to have
enough mongrels running and enough fully primed instances of postgres
up with several of the more commonly used tables up in RAM), but I hit
one performance brick wall that I am not sure how to get around.
If a user requests one of our larger queries (which can take up to 2-3
minutes to run), that mongrel is blocked as rails is chugging away
with postgres getting the resulting data set. I have tuned the query
(it now takes 2-3 minutes instead of 8-15) and have the correct
indices on the tables etc. I am sure I can do more here, but it is
diminishing on the speed returns.
The problem is not so much the response time to the user who is doing
the query as they know that this query will take time and this is
expected. It is done via an AJAX call and they get some progress
The problem is that the apache server then takes the next incoming
requests and sends them off to the mongrels in turn, it wraps around
all the mongrels and tries to serve again to this mongrel that is
doing the long query and so the second user gets blocked waiting for
the first query to finish.
One handling would be multi threaded rails app, but I am sure there is
a better option.
I tried setting the Apache balancer to max=1 but this didn't seem to solve it.
How is anyone else handling this? Running it in backgroundrb doesn't
seem to be an option because I am producing an interactive list, not a
static page / pdf or report.
So, how do I get apache to ignore this blocked mongrel and skip on to
the next one?