backgroundrb issues

rubyonrails-talk
#1

Hi there,

I'm making a site that does a lot of image processing. In order to
avoid timeouts while the clients image is being processed I have used
backgroundrb to process the images.

However, I have a few small issues:

1. Images tend to come in short, intense bursts. I want all the
images to be processed at the same time, but it seems like drb is
processing the images in sequence, with each image waiting for the
next to be processed before it starts processing. Is there any way to
make drb process them all concurrently?

2. Sometimes my worker just doesn't start. I can't find any errors
anywhere. But looking the log indicated that the worker never got the
message to start. I can't see any correlation between image size, type
or anything. It seems pretty random but happens about once every 10
images. If it was a problem with my code I would expect to see an
error message somewhere, but I can't find one. I am assuming that
something goes wrong, or gets lost, and that the error is never
logged.

Does anyone have any suggestions?

Thanks,
Jonzo.

#2

Hi Jonzo,

I'm also using backgroundrb for images processing. Mine are processing
the images one at a time - not really an issue for my application,
after all the application has returned control to the user. I'm not
getting any problems with 'bursts' of activity or missing workers btw
I'm using the latest backgroundrb. Processing time seems very linear.
Deployment OS is Centos, RMagick for processing and Mysql for DB
( images stored in filesystem).

regards
Kevin

#3

Thanks Kevin, it's good to know that it does work for some people!

The bursts of activity isn't actually a bug or problem, it's just the
way my application behaves. Typically people will upload 10 or so
images at once so I am starting that many workers. It just seems like
sometimes the workers don't start, and when one of the workers doesn't
start there is nothing printed out in any log files. no errors, and
none of the log entries I have put in my code either.

Can you see any errors in my logic/code?

Here is my worker code: