tracking sessions

If by 'active' you mean someone that has loaded a page within the last
X minutes, then you have no choice but to record every hit somewhere.
If it was me I would use memcached. On ever page hit check the cache
to see if the user exists, and if not add them using the username as
the key and whatever info you want to store as the value. Have
memcache expire them automatically after whatever time period you
consider them no longer active.
Or you can just use the database which will work fine also unless you
have a ton of traffic.

Chris

snacktime wrote the following on 14.10.2006 09:02 :

  

I wish to include a feature that will show if users are online or
offline on my site. I have a number of ideas but I am not sure of the
best way to do this in rails.

Firstly, for the record I am using a plug-in acts as authenticated.

My first idea was to set up a session model. This session model would
then be created or updated each time a page is loaded. however, this
would cost an extra SQL query every time a pager is loaded.

My second idea, was to use the rail's session variables. However, I am
not sure how to get a list of all the current session variables active
on the site. This way, I don't know how to track the movements of other
uses from any one particular user.

does anyone have any suggestions?
    
If by 'active' you mean someone that has loaded a page within the last
X minutes, then you have no choice but to record every hit somewhere.
If it was me I would use memcached. On ever page hit check the cache
to see if the user exists, and if not add them using the username as
the key and whatever info you want to store as the value. Have
memcache expire them automatically after whatever time period you
consider them no longer active.
  
The problem I see with memcache is that AFAIK it isn't queryable (is it
a proper word?). What I mean is that you can't list the content of a
memcache server farm and you certainly can't list it with SQL-like filters.

I had a similar interest and relied on the fact that you can use
ActiveRecord to store the sessions with updated_at and created_at
values. I added these two column to the sessions table and now uses the
following code to get the 'active' (used less than 30 minutes ago)
sessions count :

CGI::Session::ActiveRecordStore::Session.count [ "updated_at > ?",
30.minutes.ago.utc ]

You can use CGI::Session::ActiveRecordStore::Session to find sessions
based on the created_at/updated_at values and then use the instances
like any other ActiveRecord::Base instance. The session itself is in the
'data' attribute.

As the session is serialized in a 'data' TEXT column you can't select
sessions by their content (even a user_id) though (you need to use a
hand-made filter which stores the information you want to select on in
another table if you really need this kind of functionnality).

Lionel

Stewart wrote the following on 14.10.2006 15:29 :

Lionel Bouton wrote:
  

snacktime wrote the following on 14.10.2006 09:02 :
    

would cost an extra SQL query every time a pager is loaded.
        

X minutes, then you have no choice but to record every hit somewhere.
If it was me I would use memcached. On ever page hit check the cache
to see if the user exists, and if not add them using the username as
the key and whatever info you want to store as the value. Have
memcache expire them automatically after whatever time period you
consider them no longer active.
  

The problem I see with memcache is that AFAIK it isn't queryable (is it
a proper word?). What I mean is that you can't list the content of a
memcache server farm and you certainly can't list it with SQL-like
filters.

I had a similar interest and relied on the fact that you can use
ActiveRecord to store the sessions with updated_at and created_at
values. I added these two column to the sessions table and now uses the
following code to get the 'active' (used less than 30 minutes ago)
sessions count :

CGI::Session::ActiveRecordStore::Session.count [ "updated_at > ?",
30.minutes.ago.utc ]

You can use CGI::Session::ActiveRecordStore::Session to find sessions
based on the created_at/updated_at values and then use the instances
like any other ActiveRecord::Base instance. The session itself is in the
'data' attribute.

As the session is serialized in a 'data' TEXT column you can't select
sessions by their content (even a user_id) though (you need to use a
hand-made filter which stores the information you want to select on in
another table if you really need this kind of functionnality).

Lionel
    
I definitely like the idea of using active record to record sessions.

I still have some lingering questions about this.

Number one.
Where would I put the code that updates the session?

Nowhere, it's already in Rails. You only have to:

- create the 'sessions' table:

rake db:sessions:create

This creates a migration file in db/migrate for you. You may add the
created_at column by editing the migration fiel if you need the
information. To actually create the tabel, use rake:db:migrate.

- configure Rails to use it:

In config/environment.rb :
config.action_controller.session_store = :active_record_store

That's all.

CGI::Session::ActiveRecordStore::Session.count [ "updated_at > ?",
30.minutes.ago.utc ]
    
I am a little unsure on what this line means. This is simply
referencing a collection of session models from the database?

This returns the number of 'active' sessions by counting the one used in
the last 30 minutes. If you are confused by the 'utc', it is there
because all my timestamps are stored as utc timestamps in the database,
in config/environment.rb :

config.active_record.default_timezone = :utc

Lionel.

The problem I see with memcache is that AFAIK it isn't queryable (is it
a proper word?). What I mean is that you can't list the content of a
memcache server farm and you certainly can't list it with SQL-like filters.

You would already have the usernames in some model I assumed, and if
the username is the key then it would be easy to pull them all out of
memcached.

Chris

snacktime wrote the following on 14.10.2006 20:21 :

The problem I see with memcache is that AFAIK it isn't queryable (is it
a proper word?). What I mean is that you can't list the content of a
memcache server farm and you certainly can't list it with SQL-like filters.
    
You would already have the usernames in some model I assumed, and if
the username is the key then it would be easy to pull them all out of
memcached.

This assumes 2 things :
- one user can only have one session (not always a restriction you want
and not one you have most of the time),
- the cost of looking up the user list and querying each of them
individually isn't too high. The problem here is that memcached doesn't
scale at all for this kind of thing : you can't balance the load
associated with the client queries themselves, only the load on the
memcached servers : the more users you have, the longer you take to get
the sessions.

Lionel

It would seem to me that the simplest solution might be to add a
user_id column to the Session model and have a before_filter
:touch_session that updates the session's user_id if a user is logged
in.

Erik wrote the following on 14.10.2006 21:54 :

It would seem to me that the simplest solution might be to add a
user_id column to the Session model and have a before_filter
:touch_session that updates the session's user_id if a user is logged
in.
  
Hum, should work. For performance reasons, I wouldn't update the row
each time, but only when the user_id changes (ie: the user logs in/out).
This method could proove itself quite valuable.

my_session =
CGI::Session::ActiveRecordStore::Session::Session.find_by_session_id(_session_id)

provided you know where to look for the _session_id value (should be the
_session_id cookie value IIRC) followed by

my_session.update_attribute(:user_id, session[:user_id])

should do the trick.

Too bad the session must be serialized: this forces data to be copied
around to make it usable by standard SQL but I can't see any way of
avoiding this (without restricting what the session can store too much).

Lionel.

Why is it a concern? The user_id isn't going into the session data
column, it's going into its own column in the database table.

The schema migration might look something like:

class CreateSessions < ActiveRecord::Migration
  def self.up
    create_table :sessions do |t|
      t.column :session_id, :string, :limit => 32, :null => false
      t.column :user_id, :integer
      t.column :created_at, :datetime, :null => false
      t.column :updated_at, :datetime, :null => false
      t.column :data, :text
    end

    add_index :sessions, :session_id, :unique => true
    add_index :sessions, :user_id
    add_index :sessions, :created_at
    add_index :sessions, :updated_at
  end

  def self.down
    drop_table :sessions
  end
end

Erik wrote the following on 14.10.2006 23:19 :

  

Too bad the session must be serialized: this forces data to be copied
around to make it usable by standard SQL but I can't see any way of
avoiding this (without restricting what the session can store too much).
    
Why is it a concern? The user_id isn't going into the session data
column,

Unless you want to make (additional) ad-hoc queries to fetch the user_id
each time you need it, you'll rely on the 'session' object to store the
user_id and not a CGI::Session::ActiveRecordStore::Session instance. If
you do that, the user_id will be stored in the hash serialized in the
data column.

This is not a major problem, but you must keep in mind that you have two
places where your user_id is stored, you'll have to maintain consistency.

Lionel

Well perhaps, but only if you're not rebuilding your session when you
change credentials. I'd love to illustrate:

For instance, I wanted a more secure session store for my application.
I didn't like the fact that the default handling used only the
session_id for granting access to session data and I didn't like the
way most "drop-in" authentication systems didn't rebuild the session
during credential changes or access escalations. So I wrote my own.
In my own session handling, I use a mysql back end with a schema just
like the one I showed you earlier in the thread but it also stores the
ip address in a field I call 'host'. I wanted my sessions to use the
remote IP address of the client in addition to the session_id for
controlling access to the session storage. That would help protect my
users from session fixation a little bit, but I also didn't want to
just destroy sessions whose session_id didn't have a matching host
address. Doing that would still leave my my sessions open to denial of
service attacks. For that problem, I had to modify
ActionController::CgiRequest so I can non-destructively rebuild
sessions. I also did not want to endure the overhead of hitting the
database 2 or 3 times extra with each request just to tag each session
with the user_id and host, so I also modified CGI::Session and wrote my
own session store where you can set the host and user_id in the session
object where they will be handled during normal session reads/writes.
To illustrate, here's a piece of my controller:

require 'action_controller_ext' # my changes to ActionController
class ApplicationController < ActionController::Base
  before_filter :automatic_login_filter, :touch_session
  after_filter :touch_session

  protected
  def automatic_login_filter
    if cookies[:gwb_login_key] =~ /[a-z0-9]{40}/i
      key = LoginKey.find_by_key cookies[:gwb_login_key]
      if !key.nil? && key.host == request.remote_ip && key.active?
        self.app_user = key.user
      else
        cookies[:gwb_login_key] = nil
      end
    end
  end

  def touch_session
    # I re-wrote reset_session so that it does not kill the session,
but create a new one
    reset_session unless session.host.nil? || session.host ==
request.remote_ip
    # These next two lines set attributes in the session store and the
session store
    # will update the database normally at the end of the HTTP request.
    session.host ||= request.remote_ip
    session.user ||= session[:user_id]
  end

  # rest of the application.rb file omitted
end

I'd post my modifications to CGI::Session and
ActionController::CgiRequest and my session store code, but I'm not
really in a position to do that right this second. My inspiration for
the session store was the mysql_session_store by Stefan Kaes and it
only took me about an hour or so reading the CGI code to figure out how
to modify it to include extra columns.

Also, whenever a user in my application logs in or out, their session
is destroyed and rebuilt--so I never have to worry about a user logging
out and still showing as being "online" when I query the database. I'm
pretty satisfied with it and quite pleased with myself considering I've
only known ruby for about 3 weeks now.

The hardest part was trying to figure out why my controller tests were
failing anytime a session was rebuilt--I had to rewrite bits of the
test session store as a result, but it all works as expected now. I
didn't realize at first that a different session storage apparatus was
used during testing.

Erik

Stewart wrote the following on 15.10.2006 16:07 :

1. I have my session migration set up and I am using active record to
store all sessions

2. when a user logs in successfully the following line of code is
executed

session[:user] = current_user

if all of this works like a think it does each session should have the
entire user model stored within it.

If that's true what's the easiest way to do say an ID search on that
user model based on the global session collection.

CGI::Session::ActiveRecordStore::Session

For example if I wanted to find out if a certain user with a certain ID
is logged on.

can I do something like this?

CGI::Session::ActiveRecordStore::Session.user.Find_by_id(id)
  
No you can't. The ActiveRecord::Base#find* methods generates SQL. As the
session is stored serialized in the data column, there's no way you can
use SQL to select on the value of one of the session's keys. This is the
sole reason for the whole discussion that just took place.

Lionel.

Erik wrote the following on 15.10.2006 02:51 :

  

This is not a major problem, but you must keep in mind that you have two
places where your user_id is stored, you'll have to maintain consistency.
    
Well perhaps, but only if you're not rebuilding your session when you
change credentials. I'd love to illustrate:

For instance, I wanted a more secure session store for my application.
I didn't like the fact that the default handling used only the
session_id for granting access to session data and I didn't like the
way most "drop-in" authentication systems didn't rebuild the session
during credential changes or access escalations. So I wrote my own.
In my own session handling, I use a mysql back end with a schema just
like the one I showed you earlier in the thread but it also stores the
ip address in a field I call 'host'.

Interesting. I like having the IP in the session (not only for adding
another level of security but also for debugging). You could have used
the standard session to do that tough, there's no need for a separate
column (you get your session by _session_id value as usual and can check
for the session[:host] value in your authorize method). Anyway having
access to the host value in the table itelf has other advantages than
just another security check as I said.

Note : there could be problems with users behind proxy pools, but I'm
not sure if this is still relevant (IIRC AOL used proxy pools and you
couldn't be sure that the same user wouldn't change proxy during one
session). Anyone knows if the problem still occurs?

  I wanted my sessions to use the
remote IP address of the client in addition to the session_id for
controlling access to the session storage. That would help protect my
users from session fixation a little bit, but I also didn't want to
just destroy sessions whose session_id didn't have a matching host
address. Doing that would still leave my my sessions open to denial of
service attacks. For that problem, I had to modify
ActionController::CgiRequest so I can non-destructively rebuild
sessions. I also did not want to endure the overhead of hitting the
database 2 or 3 times extra with each request just to tag each session
with the user_id and host, so I also modified CGI::Session and wrote my
own session store where you can set the host and user_id in the session
object where they will be handled during normal session reads/writes.
  
This is a neat approach and not so complex. You change the Session
interface by adding new methods to your implementation but it isn't a
major drawback (you probably won't ever need to go back to another
implementation and at the same time can make your implementation evolve
to leverage memcache's scalability if you need to).

Lionel.

Theres not really any way to search for an active session by the user
ID in any existing "canned" rails session handling scheme--at least not
to my knowlege. It might be a very simple alternative for someone to
simply add a last_seen column to their users table and find your
"online" users by searching that column for users who accessed the site
in the last 15 minutes (or whatever duration you prefer). The drawback
there is another hit to the database with every request, but that's
really not so bad in most cases.

@Lionel:
I'll see if I can distill my session hack and post it somewhere, if you
would like to see it.

Erik

By doing that, you would have a list of active sessions, yes. However,
to find out which registered users were online (as opposed to anonymous
users), you would have to iterate the session records and check each
one for a user_id. On top of that, if you wanted to pull information
about each user that you found logged in (like user name, email
address, whatever you might have in the users table), you would have to
query the database again for each active session with a user_id--which
can quickly add up to a LOT of extra database hits and consequential
slow-down.

Erik

I posted on my blog about the session store I am currently using (the
one I was describing above). Most of my source code for the session
(if not all of it) is on display there.

http://burningtimes.net/articles/2006/10/15/paranoid-rails-session-storage

Erik

Erik wrote the following on 16.10.2006 13:23 :

I posted on my blog about the session store I am currently using (the
one I was describing above). Most of my source code for the session
(if not all of it) is on display there.

http://burningtimes.net/articles/2006/10/15/paranoid-rails-session-storage

Nice documentation ! I've not read everything yet but this seems well done.

Thanks,

Lionel

Nobody has criticism to offer me?

Erik