Cron job in single environment


We have an app running that imports near realtime values from a feed. We did that using BackgroundRb, but I was pointed at the fact that BackgroundRb has quite the memory footprint.

So I switched to a rake task using a regular Linux cronjob. However, the feed we're using only accepts one concurrent connection from our (whitelisted) IP at a time. With BackgroundRb I'd just initialize a single connection in the create method of the worker and make every worker instance use that connection. I'd only reinitialize the connection when it isn't active anymore.

With rake, however, I can't seem to do such a thing. I tried setting a global $connection in an initializer and checking its state on every call of the rake task. If active -> use, if not -> reinitialize. It doesn't work because every rake task seems to fire off a new instance of the app. If the rake tasks overlap (which they do from time to time), the second rake task will fail because the connection is already open.

How would you approach this? Can this be done using a rake task? Or do we need to switch to another cronjob-like system?

Regards, Jaap Haagmans