Multiple workling instances

I actually just was in the same situation. I also had thought multiple Starling instances would help things, but it didn't. The bottleneck was Workling, and I imagine it is the same for you. Despite the lack of documentation, it turns out to be extremely easy to run several instances:

1) In script/workling_client.rb, change :multiple from false to true as you already have, so the Daemon knows to allow for multiple instances 2) Just start more instances by calling "script/workling_client start" several times. It will create workling0.pid, workling1.pid, etc. I'm not sure how to properly setup god.rb to monitor the multiple instances so if you do please let me know :slight_smile:

FYI, you could also create multiple worker classes (my_worker.rb, another_worker.rb etc), and each Workling instance would use a thread for each class. So if you split up your methods into several classes, it might help things even more.

PS. Here is an extremely helpful series of blog posts for starling and workling: http://davedupre.com/2008/03/25/ruby-background-tasks-with-starling/

I have modified my workling_client script to better handle multiple worklings. This approach gives you full control over the workling process names and pid files, which you can then use to cleanly manage the worklings using God or Monit.

See below for my (admittedly hacky) solution:

#!/usr/bin/env ruby require 'rubygems' require 'daemons'

# This is a bit hacky, but we don't have a clean way to hook into the parsed # options, as Daemons is managing these internally. number = 0 for i in 0..ARGV.length   if ARGV[i] == '--number'     number = ARGV[i + 1]     2.times {ARGV.delete_at(i)}   end end

puts "Workling Number: #{number}"

workling = File.join(File.dirname(__FILE__), '..', 'vendor', 'plugins', 'workling', 'script', 'listen.rb') options = {   :app_name => "workling-#{number}-client",   :ARGV => ARGV,   :dir_mode => :normal,   :dir => File.join(File.dirname(__FILE__), '..', 'log'),   :log_output => true,   :multiple => false,   :backtrace => true,   :monitor => false }

Daemons.run(workling, options)