Is it really that hard?
I'm not sure what exactly it is you think it's hard. Backgroundrb provides a pretty simple interface to starting worker tasks.
In a java servlets app I'd just have something in my servlet's init method to start a thread pool, and use syncronized queues etc. to share the work. Or if I wanted more complex scheduling I'd use Quartz - either way, everything is in the same memory space, no external communications, no marshalling of objects...
So, why can't I do something similar in Rails? Or is there some obvious thing I'm missing?
Rails is not threadsafe[1]. Does this answer your question?
[1]: http://blog.dgibbons.net/articles/2006/08/08/why-you-need-multiple-mongrel-instances-with-rails