Rails way to do long running tasks

rubyonrails-talk
1

Hello Rails community,

  I would like to get some advices - I am writing a web application that will be sending some emails and execute other long running tasks.

  Some of these tasks may take some time and I do not want the user experience to be impacted. I would appreciate any pointers on advices on these areas:

   * how to send emails without making the user wait    * how to execute long running tasks    * The best way to do periodic cron jobs in Rails

Thank you for your help in advance.

Chris

2

Chris wrote:

   * how to send emails without making the user wait

Either Thread.new{}, or a plugin called Spawn. You need the latter if your e-mails call templates which in turn call ActiveRecord. A separate thread requires a separate database connection, and Spawn handles this.

   * how to execute long running tasks

The threading option has one flaw: If your web server expunges your Passenger module from memory, the thread lapses.

When sending a few emails, if you can't send them before process harvesting time, you have bigger problems than a missing email. Failing that analysis, you need an out-of-process solution like BackgroundRB. It runs in a daemon of its own and communicates with your app thru dRB. In exchange for a fatter and more fragile implementation, you get a process you can control directly. However...

   * The best way to do periodic cron jobs in Rails

Create a folder called cron, write (using TDD!) a script that does what you need, and install it into your OS's cron system as the command line script/runner lib/my_script.rb.

If I needed to send e-mails, I would pool them up, then push them out with a cron. The previous two options allow communication back to the user in subsequent controller actions. You probably don't need that, so just configure a cron to run every 90 seconds, and send emails for up to 1 minute. Take a little care to not send the same email twice, and you are set.

3

For emails, the simplest way is to set up a local MTA (postfix, for instance) and send the mail to that. The MTA takes care of the hard part delivering the mail as needed.

For more general things, there are some gems that let you set up cron- like tasks, or you can just call a rake task from plain old cron.

--Matt Jones

4

Two pointers from personal experience:

1. Don't use backgroundrb for asynchronous processing 2. Do use javan-whenever to automate your cron jobs (available on Github)

Bharat

5

@Bharat,

Can you give brief why to use javan instead backgroundrb ? As, i am using backgroundrb for heavy upload duties and it works fine for me.

  • Sandip R~
6

Thanks a lot for the pointers and the great suggestions.

I really do find BackgrounDrb too heavy for my implementation - and think the gems you described make sense.

Spawn plugin and Javan-whenever is great. And there is a Railscast that is really good in explaining: #164 Cron in Ruby - RailsCasts

7

I would highly recommend using delayed_job (http://github.com/tobi/delayed_job/tree/master). We’ve recently integrated this into the BioCatalogue (http://www.biocatalogue.org) to great effect.

This is what GitHub use I believe (http://github.com/blog/197-the-new-queue).

Useful blog post that walks you through setting up delayed_job - http://www.magnionlabs.com/2009/2/28/background-job-processing-in-rails-with-delayed_job

Useful FAQ: http://wiki.github.com/tobi/delayed_job

Examples: http://github.com/igal/rubyqueues/tree/master/delayed_job_eg/

Comparisons: http://github.com/igal/rubyqueues/raw/802998678c012ed17b27324a41a4cd42a377a678/comparison.html

Hope that helps.

Jits