server overload

Hi,
i've got an RoR site, which runs on mongrel
there is a task which the admin uses to fill the database with new
data, the problem is that when the task runs the whole webpage
stalls,

the job he does from the admin panel also runs as a cron job at a time
the page isn't used but he insists for the manual run.

So i don't know if this is a rail problem or mongrel/this whatever -
but would be very grateful for a help

adam

How much data is being loaded? If you are adding a large amount of
data inside a giant commit statement then the database could become
inaccessible for everyone else until the load completes.

Worse than that, IIRC, Mongrel serves one request at a time. If your
request takes a long time, *nothing* else will respond while it's
processing.
Consider a Mongrel cluster (so there are other Mongrel instances to
handle other requests) or better yet, switch to Passenger with Apache
(or NGinx, etc)

well its from an xml file which usually is about ~20MB i found one
solution which isnt very pretty to run the web page on two different
ports using two different instances of mongrel, but I'm looking for
somthing more simple for the end user, so he dosn't have to remember
to login to one webpage for the admin work.

Yeh i thought about mongrel cluser - I'm just imaging the discussion
with the server admins :wink:

Use a delayed_job task for the long running task.

This will free your mongrel process for further requests as delayed job runs as a separate ruby process but loads the rails environment. You can use “send_later” for this and this requires minimal code change.

  • Gautam
    @gautamrege

this sound very interesting I'll check that out

While delayed_job is handy, you will eventually run out of steam if
you only have one mongrel. Others have suggested mongrel_cluster for
running multiple mongrels, you might also want to consider passenger,
which will fork extra rails instances as needed.

Fred

I do agree - infact my preferred setup us Nginx+Passenger.

However, given that the current problem on hand is one long running task slowing things down and not a scale issue (I.e number of requests), delayed_job may just do the trick.

My 2 cents :slight_smile:
- Gautam

www.joshsoftware.com

I'm currently trying to set up the delayed job solution, running into
some problems
but if I get it to work I think it will do the job, the importing will
be done about once a day.

Since I'm on the topic I've tried to do the railscast setup from
delayed job but both of the versions showed there dont do it for me
the first one with the send_later method
Download.new.send_later(:start), shows "undefined method `perform' for
#<YAML::Object: ..."
I load the config in environment.rb file (raw_config =
File.read(RAILS_ROOT + "/config/config.yml")
APP_CONFIG = YAML.load(raw_config)[RAILS_ENV])

the second option using a new class after jobs:work

"Job failed to load: Unknown handler. Try to manually require the
appropriate file."

Adam,

I presume you have run the migration that is generated by delayed_job. You also have the initializer file.

Earlier would you start the long running as Download.new.start ? If so, Download.new.send_later(:start) would definitely work.

If ‘start’ is an instance method in the Download model, it will work. If ‘start’ is a class method, you cannot (obviously) do a Download.new.start – then you can enqueue the task.

Send some more details - and I can help you sort this out.

  • Gautam

@gautamrege

hmm, I've played around a bit and it somehow started to work well I
guess thats good just wish how I've done that so I could setup it on
the server properly :wink: