communicating between multiple rails applications?

Short story:

My company has a rails application as our product. As we add more functionality, we are thinking of building new small, self-contained rails applications rather than just chugging along with our single monolithic application that we have right now.

What is the best way to do this? Is it even recommended?

Long story:

My company, www.readMedia.com, specializes in press release distribution over the web and our product is a 60,000 line Ruby on rails application. We want to start adding some functionality that sort of has to do with existing functionality, but not really. It would use most of the same data, but the product could be sold separately to new clients, or as an add-on to old clients.

The easiest thing to do would be to simply build that functionality into our existing application. However, that has some drawbacks. As the application gets bigger and bigger, and we hire more programmers, the testing gets harder, the organization gets harder, deployment gets harder, the code gets messier, et cetera.

What we would like to do instead is to build new functionality as a separate rails application entirely. This has many advantages. Each application/functionality can be upgraded independently, turned on or off at will, have its own team of programmers who never have to worry about stepping on the feet of other teams, source control becomes easier, deployment is easier, et cetera.

The problem, to me, is that this seems to be technically really REALLY hard, if not impossible. It sounds wonderful in theory, but can (should) it even be done? Although the functionality of each application would be different, they would share most of the same data, such as our geographic table of zip codes and counties and our history of sent releases.

One thought is to use the same database for each application. This seems really tough. How would we keep the models in sync with each other, because things such as validation is very important? Another thought is to use multiple databases, one with the shared information, and another for each application to hold its own stuff. Again, very difficult. As far as I can tell, rails simply can't use multiple databases. I've found a plug-in or two that supposedly does this, but I'm not sure I would trust it.

Another thought is to have some API between the old system and the new applications that we are making. Although certainly possible, this seems like an extraordinary amount of work. How do we secure communication between applications? How do we share user accounts? Making an API would basically mean re-creating all of the existing functionality in an outward facing manner that can be called from another application. I don't really know how to do that, nor do I have any experience with that sort of thing.

Another thought is to synchronize the data occasionally between our applications. Have a background task that runs every night and updates each database in sync with each other. This is a nice solution, but it means that changes are propagated slowly, and syncing has historically been very difficult in the programming world.

As my final thought, I've been thinking about what 37 signals does. Their applications don't really interact at all. You have to login separately to each one, data isn't shared between them, they are basically separate products. In our situation we would like our products to be more tightly intertwined than the 37 signals ones. However, given that 37 signals doesn't do it, perhaps it's not possible? It's not what rails was designed to do? It's a bad idea?

Anyway, what do all of you think? Do we just keep growing and growing the main application? Do we force ourselves to create completely separate products?

If we can somehow segment functionality into different rails applications that still work together as a whole, what is the best way to do it? Has anybody tried? Does anybody have any experience with this? Please share any tools, plug-ins, or best practices that you think could help.

Thanks a lot!

That's basically what we do. Separate apps have separate databases and never touch the database of another app (so it's dead easy to isolate an app (eg if it needs more horse power than the others or if it is criticial) by just running it off a different database server. Apps usually communicate with each other via private APIs over http. Depending on how your app works you may find that ActiveResource is useful, most of our interapplication communications don't really fit with that model and so we haven't used that much

I would avoid synchronizing data, sounds like a headache (the only place we do this is with our single sign on system, so that apps can operate when it isn't available)

You may be wondering what to do when you want to display data from two apps on the same page. The way we do it is to have javascript objects on the page that fetch content from the different apps via ajax. A single page may contain fragments of html rendered by any number of applications.

My colleague and myself presented on this subject at railsconf europe 2008. The slides should still be available, as well as demo code used in the presentation. Both should be linked from

Fred

Thanks! I read your presentation and some of your source code. You guys definitely planned well from the start. It will be a little bit harder for us without any planning, but it's definitely doable. I had no idea that ActiveResource was so awesome!

We were in a very similar place to you not so far back, with just 1 big app so don't lose hope!

Fred