Session state management in Lighttpd-mongrel cluster

You can't. You need some type of load balancer that can route
requests from the same ip addresses to the same mongrel on subsequent
requests. Yes it's a pain, but if you want to have clusters on
different servers and use the local filesystem for session storage you
don't really have another option. I think you can do this with
something like squid. I've always used something like the serverirons
for this type of stuff, but I know there are open source solutions for
it.

A second best approach would probably be mysql. It should be pretty
fast for session storage if configured correctly.

Excuse me for my ignorance but I am not sure if ActiveRecordStore for
session would be faster than file system session store?

It's most likely not.

BTW its really surprising that there is no rails setup for a transparent
failover, the idea of which can be sold to large corporations like mine.
I think this is a common requirement nowadays when most web applications
require 99.95% uptime. I am not so highly qualified to take on such a
task, but unless issues like this are tackled RoR adoption rate in
corporations will be extrememly low at best.

What you want to do isn't a function of the application server,
whether it's rails, mod perl, jboss, whatever.. Load balancing and
failover is handled by other layers. Just because you aren't familiar
with the different tools available to you dont' assume they don't
exist. I already pointed three that would probably do the job.
Squid, Pound, or something like a ServerIron, and there are more if
you look around.

We are working on high availability redundant rails clustered hosting. http://engineyard.com . It will be ready for beta in early October.

Cheers-
-Ezra

        We are working on high availability redundant rails clustered
hosting. http://engineyard.com . It will be ready for beta in early
October.

Will be looking forward to seeing that when it's out, definitely seems
like a void that needs filling.

> Excuse me for my ignorance but I am not sure if ActiveRecordStore for
> session would be faster than file system session store?
>
It's most likely not.

And, to speak with someone elses words: that's not a problem until it's
a problem.

Once the AR session store proves to be a bottleneck, you can always
replace it with a faster, still MySQL-based solution (afair the Rails
performance blog had a post about this some time ago). And if that still
doesn't work out, you can go back to solve the problem by choosing
another session store, or even by implementing your own
shared-fs-storage-safe one.

> BTW its really surprising that there is no rails setup for a transparent
> failover, the idea of which can be sold to large corporations like mine.
> I think this is a common requirement nowadays when most web applications
> require 99.95% uptime.
> I am not so highly qualified to take on such a
> task, but unless issues like this are tackled RoR adoption rate in
> corporations will be extrememly low at best.

What you want to do isn't a function of the application server,
whether it's rails, mod perl, jboss, whatever.. Load balancing and
failover is handled by other layers. Just because you aren't familiar
with the different tools available to you dont' assume they don't
exist. I already pointed three that would probably do the job.
Squid, Pound, or something like a ServerIron, and there are more if
you look around.

Pen can do session aware load balancing, too. In fact it's
not really session-aware, it simply assigns same clients (IP-based, I
suppose), the same Mongrel instance with each request.

Don't know about Windows support, though.

regards,
Jens

What you are referring to is same/similar to proxy.balance="hash" option
of lighttpd, but I still can't understand how it will help with session
state management in case of failover. Lets say Pen/Lighty assigs the a
mongrel instance to all requests from the same IP, if this mongrel
instance goes down, what happens to the session data and where will the
next request from the same IP go?

To another server. If you use sessions wisely and don't just store
the whole kitchen sink in them, then losing a few now and then is not
going to be that big of a deal. So once a year a server dies and 10%
of your users find themselves logged out or their shopping carts gone.
Some things are an acceptable risk.

I agree that some risks are acceptable and this may be the case for
80-90% of apps, but that was not what I as looking for.

Risks involved in loosing the session state can be very high depending
on the critical nature of the application, for instance loosing
transaction data due to session failure if this happens in a e-commerce
application will be disastrous.

If you store that kind of information in sessions you are nuts.
That's not what sessions are for. If you insist on doing that, I'm
not sure what to say other than you need to seriously rethink your
design.

Coming back to our original subject, when we talk about load balancer
like lighty, Pen or Pound I don't see any of them managing the session
synchronization between various nodes in the cluster. So basically none
of the currenty existing solutions provide transparent failover
capabilities like BEA Weblogic or IBM's Websphere. This is one area
where RoR so far is behind the technology curve, but hopefully somebody
is already working on it.

Your requirements were for file based sessions. I highly doubt that
IBM or BEA can provide truly transparent failover under those
conditions. And like I said before, there are other solutions such as
squid, mod backhand, or hardware based solutions like BigIP or
ServerIrons. BEA or Websphere don't do anything you couldn't do with
rails, mod perl, zope, or other platforms. What you dont' realize is
that those commercial solutions simply incorporate the other layers
into one single solution, but it's still all the same. With the open
source solution you just need to put the pieces together on your own.
The open source approach takes more knowledge. You do need to have a
good grasp of what tools are available and how to use them.

Wait, this isn't correct *at all*.

Rails currently provides, behind all web servers including Apache, Lighty,
Pen, Pound, Mongrel, Litespeed, hell even WEBrick(!) several solutions for
handling session in failover conditions with 100% alacrity.

1) Shared filesystem
2) DB (ActiveRecordStore and SQLSessionStore)
3) Memcached

Where do you think Weblogic and Websphere store the session data?