Best way to implement background job processing

rubyonrails-talk
,
1

Hi there!

I have this situation: In my production server, I have 24 RoR apps running and it will be increasing as time goes by. Each of them has its own background job logics (for AS file processing, email sending and mailbox handling).

It seems that there are a couple of memory leaks every once in a while due to memory not being freed after job processing.

So my question is: How can I effectively implement a background job queue and which ones are preferred to handle this situation? I’m currently using backburner with beanstalkd, but it seems that I’m using quite a lot of resources for processing my jobs, so I’d like to evaluate the possibility for changing it for another one.

Thanks in advance for all your help!

2

Have you figured out what is causing the memory leak and tried to address that directly? Changing job back-ends is a drastic measure and may not solve your issue and/or may create new issues. A lot of times memory leaks are the result of the app and not the framework, so switching frameworks may not fix it if you don’t know the source of the leak

1 Like