Session state management in Lighttpd-mongrel cluster

You can't. You need some type of load balancer that can route requests from the same ip addresses to the same mongrel on subsequent requests. Yes it's a pain, but if you want to have clusters on different servers and use the local filesystem for session storage you don't really have another option. I think you can do this with something like squid. I've always used something like the serverirons for this type of stuff, but I know there are open source solutions for it.

A second best approach would probably be mysql. It should be pretty fast for session storage if configured correctly.

Excuse me for my ignorance but I am not sure if ActiveRecordStore for session would be faster than file system session store?

It's most likely not.

BTW its really surprising that there is no rails setup for a transparent failover, the idea of which can be sold to large corporations like mine. I think this is a common requirement nowadays when most web applications require 99.95% uptime. I am not so highly qualified to take on such a task, but unless issues like this are tackled RoR adoption rate in corporations will be extrememly low at best.

What you want to do isn't a function of the application server, whether it's rails, mod perl, jboss, whatever.. Load balancing and failover is handled by other layers. Just because you aren't familiar with the different tools available to you dont' assume they don't exist. I already pointed three that would probably do the job. Squid, Pound, or something like a ServerIron, and there are more if you look around.

We are working on high availability redundant rails clustered hosting. http://engineyard.com . It will be ready for beta in early October.

Cheers- -Ezra

        We are working on high availability redundant rails clustered hosting. http://engineyard.com . It will be ready for beta in early October.

Will be looking forward to seeing that when it's out, definitely seems like a void that needs filling.

> Excuse me for my ignorance but I am not sure if ActiveRecordStore for > session would be faster than file system session store? > It's most likely not.

And, to speak with someone elses words: that's not a problem until it's a problem.

Once the AR session store proves to be a bottleneck, you can always replace it with a faster, still MySQL-based solution (afair the Rails performance blog had a post about this some time ago). And if that still doesn't work out, you can go back to solve the problem by choosing another session store, or even by implementing your own shared-fs-storage-safe one.

> BTW its really surprising that there is no rails setup for a transparent > failover, the idea of which can be sold to large corporations like mine. > I think this is a common requirement nowadays when most web applications > require 99.95% uptime. > I am not so highly qualified to take on such a > task, but unless issues like this are tackled RoR adoption rate in > corporations will be extrememly low at best.

What you want to do isn't a function of the application server, whether it's rails, mod perl, jboss, whatever.. Load balancing and failover is handled by other layers. Just because you aren't familiar with the different tools available to you dont' assume they don't exist. I already pointed three that would probably do the job. Squid, Pound, or something like a ServerIron, and there are more if you look around.

Pen can do session aware load balancing, too. In fact it's not really session-aware, it simply assigns same clients (IP-based, I suppose), the same Mongrel instance with each request.

Don't know about Windows support, though.

regards, Jens

What you are referring to is same/similar to proxy.balance="hash" option of lighttpd, but I still can't understand how it will help with session state management in case of failover. Lets say Pen/Lighty assigs the a mongrel instance to all requests from the same IP, if this mongrel instance goes down, what happens to the session data and where will the next request from the same IP go?

To another server. If you use sessions wisely and don't just store the whole kitchen sink in them, then losing a few now and then is not going to be that big of a deal. So once a year a server dies and 10% of your users find themselves logged out or their shopping carts gone. Some things are an acceptable risk.

I agree that some risks are acceptable and this may be the case for 80-90% of apps, but that was not what I as looking for.

Risks involved in loosing the session state can be very high depending on the critical nature of the application, for instance loosing transaction data due to session failure if this happens in a e-commerce application will be disastrous.

If you store that kind of information in sessions you are nuts. That's not what sessions are for. If you insist on doing that, I'm not sure what to say other than you need to seriously rethink your design.

Coming back to our original subject, when we talk about load balancer like lighty, Pen or Pound I don't see any of them managing the session synchronization between various nodes in the cluster. So basically none of the currenty existing solutions provide transparent failover capabilities like BEA Weblogic or IBM's Websphere. This is one area where RoR so far is behind the technology curve, but hopefully somebody is already working on it.

Your requirements were for file based sessions. I highly doubt that IBM or BEA can provide truly transparent failover under those conditions. And like I said before, there are other solutions such as squid, mod backhand, or hardware based solutions like BigIP or ServerIrons. BEA or Websphere don't do anything you couldn't do with rails, mod perl, zope, or other platforms. What you dont' realize is that those commercial solutions simply incorporate the other layers into one single solution, but it's still all the same. With the open source solution you just need to put the pieces together on your own. The open source approach takes more knowledge. You do need to have a good grasp of what tools are available and how to use them.

Wait, this isn't correct *at all*.

Rails currently provides, behind all web servers including Apache, Lighty, Pen, Pound, Mongrel, Litespeed, hell even WEBrick(!) several solutions for handling session in failover conditions with 100% alacrity.

1) Shared filesystem 2) DB (ActiveRecordStore and SQLSessionStore) 3) Memcached

Where do you think Weblogic and Websphere store the session data?