Multiple workling instances

I actually just was in the same situation. I also had thought
multiple Starling instances would help things, but it didn't. The
bottleneck was Workling, and I imagine it is the same for you.
Despite the lack of documentation, it turns out to be extremely easy
to run several instances:

1) In script/workling_client.rb, change :multiple from false to true
as you already have, so the Daemon knows to allow for multiple
instances
2) Just start more instances by calling "script/workling_client start"
several times. It will create workling0.pid, workling1.pid, etc. I'm
not sure how to properly setup god.rb to monitor the multiple
instances so if you do please let me know :slight_smile:

FYI, you could also create multiple worker classes (my_worker.rb,
another_worker.rb etc), and each Workling instance would use a thread
for each class. So if you split up your methods into several classes,
it might help things even more.

PS. Here is an extremely helpful series of blog posts for starling
and workling: http://davedupre.com/2008/03/25/ruby-background-tasks-with-starling/

I have modified my workling_client script to better handle multiple
worklings. This approach gives you full control over the workling
process names and pid files, which you can then use to cleanly manage
the worklings using God or Monit.

See below for my (admittedly hacky) solution:

#!/usr/bin/env ruby
require 'rubygems'
require 'daemons'

# This is a bit hacky, but we don't have a clean way to hook into the
parsed
# options, as Daemons is managing these internally.
number = 0
for i in 0..ARGV.length
  if ARGV[i] == '--number'
    number = ARGV[i + 1]
    2.times {ARGV.delete_at(i)}
  end
end

puts "Workling Number: #{number}"

workling = File.join(File.dirname(__FILE__), '..', 'vendor',
'plugins', 'workling', 'script', 'listen.rb')
options = {
  :app_name => "workling-#{number}-client",
  :ARGV => ARGV,
  :dir_mode => :normal,
  :dir => File.join(File.dirname(__FILE__), '..', 'log'),
  :log_output => true,
  :multiple => false,
  :backtrace => true,
  :monitor => false
}

Daemons.run(workling, options)