backgroundrb workers won't die!

rubyonrails-talk
1

Hi,

I'm using backgroundrb workers to do a lot of work that is being unloaded to a different system.. it works, but over a period of time with much usage the server my rails app is on gets overloaded with backgroundrb workers that didn't get killed. this eventually causes backgroundrb to stop working and my server takes a massive performance hit with all the memory getting eaten up.. i've heard about a trick where instead of doing doing this:

          key = MiddleMan.new_worker(:class => :myapp_worker)           worker = MiddleMan.worker(key)           worker.do_work           worker.delete

where i make a new worker in my controller each time i need it to do something.. that instead i can have a single worker created and just use that one worker throughout my rails app..

so how do i do that?

Thanks,

Stuart

2

Hi Stuart,

stewbawka wrote:

with much usage the server my rails app is on gets overloaded with backgroundrb workers that didn't get killed.

Are you sure the workers aren't getting killed? I mean, do you see the worker process pids? I ask because the code you posted didn't disconnect from the database and memory leaks due to not closing db connections is a known problem when using BackgroundRb.

If you want to test it out, use these in your worker to kill it off.

ActiveRecord::Base.connection.disconnect! ::BackgrounDRb::MiddleMan.instance.delete_worker @_job_key

hth, Bill

3

Starting only one instance of worker for the same work is a good idea, you just pass different args, but it has some limitations eg. you cannot run two tasks simultaneously and worker should be stateless.