tracking sessions

If by 'active' you mean someone that has loaded a page within the last X minutes, then you have no choice but to record every hit somewhere. If it was me I would use memcached. On ever page hit check the cache to see if the user exists, and if not add them using the username as the key and whatever info you want to store as the value. Have memcache expire them automatically after whatever time period you consider them no longer active. Or you can just use the database which will work fine also unless you have a ton of traffic.

Chris

snacktime wrote the following on 14.10.2006 09:02 :

  

I wish to include a feature that will show if users are online or offline on my site. I have a number of ideas but I am not sure of the best way to do this in rails.

Firstly, for the record I am using a plug-in acts as authenticated.

My first idea was to set up a session model. This session model would then be created or updated each time a page is loaded. however, this would cost an extra SQL query every time a pager is loaded.

My second idea, was to use the rail's session variables. However, I am not sure how to get a list of all the current session variables active on the site. This way, I don't know how to track the movements of other uses from any one particular user.

does anyone have any suggestions?      If by 'active' you mean someone that has loaded a page within the last X minutes, then you have no choice but to record every hit somewhere. If it was me I would use memcached. On ever page hit check the cache to see if the user exists, and if not add them using the username as the key and whatever info you want to store as the value. Have memcache expire them automatically after whatever time period you consider them no longer active.    The problem I see with memcache is that AFAIK it isn't queryable (is it a proper word?). What I mean is that you can't list the content of a memcache server farm and you certainly can't list it with SQL-like filters.

I had a similar interest and relied on the fact that you can use ActiveRecord to store the sessions with updated_at and created_at values. I added these two column to the sessions table and now uses the following code to get the 'active' (used less than 30 minutes ago) sessions count :

CGI::Session::ActiveRecordStore::Session.count [ "updated_at > ?", 30.minutes.ago.utc ]

You can use CGI::Session::ActiveRecordStore::Session to find sessions based on the created_at/updated_at values and then use the instances like any other ActiveRecord::Base instance. The session itself is in the 'data' attribute.

As the session is serialized in a 'data' TEXT column you can't select sessions by their content (even a user_id) though (you need to use a hand-made filter which stores the information you want to select on in another table if you really need this kind of functionnality).

Lionel

Stewart wrote the following on 14.10.2006 15:29 :

Lionel Bouton wrote:   

snacktime wrote the following on 14.10.2006 09:02 :     

would cost an extra SQL query every time a pager is loaded.         

X minutes, then you have no choice but to record every hit somewhere. If it was me I would use memcached. On ever page hit check the cache to see if the user exists, and if not add them using the username as the key and whatever info you want to store as the value. Have memcache expire them automatically after whatever time period you consider them no longer active.   

The problem I see with memcache is that AFAIK it isn't queryable (is it a proper word?). What I mean is that you can't list the content of a memcache server farm and you certainly can't list it with SQL-like filters.

I had a similar interest and relied on the fact that you can use ActiveRecord to store the sessions with updated_at and created_at values. I added these two column to the sessions table and now uses the following code to get the 'active' (used less than 30 minutes ago) sessions count :

CGI::Session::ActiveRecordStore::Session.count [ "updated_at > ?", 30.minutes.ago.utc ]

You can use CGI::Session::ActiveRecordStore::Session to find sessions based on the created_at/updated_at values and then use the instances like any other ActiveRecord::Base instance. The session itself is in the 'data' attribute.

As the session is serialized in a 'data' TEXT column you can't select sessions by their content (even a user_id) though (you need to use a hand-made filter which stores the information you want to select on in another table if you really need this kind of functionnality).

Lionel      I definitely like the idea of using active record to record sessions.

I still have some lingering questions about this.

Number one. Where would I put the code that updates the session?

Nowhere, it's already in Rails. You only have to:

- create the 'sessions' table:

rake db:sessions:create

This creates a migration file in db/migrate for you. You may add the created_at column by editing the migration fiel if you need the information. To actually create the tabel, use rake:db:migrate.

- configure Rails to use it:

In config/environment.rb : config.action_controller.session_store = :active_record_store

That's all.

CGI::Session::ActiveRecordStore::Session.count [ "updated_at > ?", 30.minutes.ago.utc ]      I am a little unsure on what this line means. This is simply referencing a collection of session models from the database?

This returns the number of 'active' sessions by counting the one used in the last 30 minutes. If you are confused by the 'utc', it is there because all my timestamps are stored as utc timestamps in the database, in config/environment.rb :

config.active_record.default_timezone = :utc

Lionel.

The problem I see with memcache is that AFAIK it isn't queryable (is it a proper word?). What I mean is that you can't list the content of a memcache server farm and you certainly can't list it with SQL-like filters.

You would already have the usernames in some model I assumed, and if the username is the key then it would be easy to pull them all out of memcached.

Chris

snacktime wrote the following on 14.10.2006 20:21 :

The problem I see with memcache is that AFAIK it isn't queryable (is it a proper word?). What I mean is that you can't list the content of a memcache server farm and you certainly can't list it with SQL-like filters.      You would already have the usernames in some model I assumed, and if the username is the key then it would be easy to pull them all out of memcached.

This assumes 2 things : - one user can only have one session (not always a restriction you want and not one you have most of the time), - the cost of looking up the user list and querying each of them individually isn't too high. The problem here is that memcached doesn't scale at all for this kind of thing : you can't balance the load associated with the client queries themselves, only the load on the memcached servers : the more users you have, the longer you take to get the sessions.

Lionel

It would seem to me that the simplest solution might be to add a user_id column to the Session model and have a before_filter :touch_session that updates the session's user_id if a user is logged in.

Erik wrote the following on 14.10.2006 21:54 :

It would seem to me that the simplest solution might be to add a user_id column to the Session model and have a before_filter :touch_session that updates the session's user_id if a user is logged in.    Hum, should work. For performance reasons, I wouldn't update the row each time, but only when the user_id changes (ie: the user logs in/out). This method could proove itself quite valuable.

my_session = CGI::Session::ActiveRecordStore::Session::Session.find_by_session_id(_session_id)

provided you know where to look for the _session_id value (should be the _session_id cookie value IIRC) followed by

my_session.update_attribute(:user_id, session[:user_id])

should do the trick.

Too bad the session must be serialized: this forces data to be copied around to make it usable by standard SQL but I can't see any way of avoiding this (without restricting what the session can store too much).

Lionel.

Why is it a concern? The user_id isn't going into the session data column, it's going into its own column in the database table.

The schema migration might look something like:

class CreateSessions < ActiveRecord::Migration   def self.up     create_table :sessions do |t|       t.column :session_id, :string, :limit => 32, :null => false       t.column :user_id, :integer       t.column :created_at, :datetime, :null => false       t.column :updated_at, :datetime, :null => false       t.column :data, :text     end

    add_index :sessions, :session_id, :unique => true     add_index :sessions, :user_id     add_index :sessions, :created_at     add_index :sessions, :updated_at   end

  def self.down     drop_table :sessions   end end

Erik wrote the following on 14.10.2006 23:19 :

  

Too bad the session must be serialized: this forces data to be copied around to make it usable by standard SQL but I can't see any way of avoiding this (without restricting what the session can store too much).      Why is it a concern? The user_id isn't going into the session data column,

Unless you want to make (additional) ad-hoc queries to fetch the user_id each time you need it, you'll rely on the 'session' object to store the user_id and not a CGI::Session::ActiveRecordStore::Session instance. If you do that, the user_id will be stored in the hash serialized in the data column.

This is not a major problem, but you must keep in mind that you have two places where your user_id is stored, you'll have to maintain consistency.

Lionel

Well perhaps, but only if you're not rebuilding your session when you change credentials. I'd love to illustrate:

For instance, I wanted a more secure session store for my application. I didn't like the fact that the default handling used only the session_id for granting access to session data and I didn't like the way most "drop-in" authentication systems didn't rebuild the session during credential changes or access escalations. So I wrote my own. In my own session handling, I use a mysql back end with a schema just like the one I showed you earlier in the thread but it also stores the ip address in a field I call 'host'. I wanted my sessions to use the remote IP address of the client in addition to the session_id for controlling access to the session storage. That would help protect my users from session fixation a little bit, but I also didn't want to just destroy sessions whose session_id didn't have a matching host address. Doing that would still leave my my sessions open to denial of service attacks. For that problem, I had to modify ActionController::CgiRequest so I can non-destructively rebuild sessions. I also did not want to endure the overhead of hitting the database 2 or 3 times extra with each request just to tag each session with the user_id and host, so I also modified CGI::Session and wrote my own session store where you can set the host and user_id in the session object where they will be handled during normal session reads/writes. To illustrate, here's a piece of my controller:

require 'action_controller_ext' # my changes to ActionController class ApplicationController < ActionController::Base   before_filter :automatic_login_filter, :touch_session   after_filter :touch_session

  protected   def automatic_login_filter     if cookies[:gwb_login_key] =~ /[a-z0-9]{40}/i       key = LoginKey.find_by_key cookies[:gwb_login_key]       if !key.nil? && key.host == request.remote_ip && key.active?         self.app_user = key.user       else         cookies[:gwb_login_key] = nil       end     end   end

  def touch_session     # I re-wrote reset_session so that it does not kill the session, but create a new one     reset_session unless session.host.nil? || session.host == request.remote_ip     # These next two lines set attributes in the session store and the session store     # will update the database normally at the end of the HTTP request.     session.host ||= request.remote_ip     session.user ||= session[:user_id]   end

  # rest of the application.rb file omitted end

I'd post my modifications to CGI::Session and ActionController::CgiRequest and my session store code, but I'm not really in a position to do that right this second. My inspiration for the session store was the mysql_session_store by Stefan Kaes and it only took me about an hour or so reading the CGI code to figure out how to modify it to include extra columns.

Also, whenever a user in my application logs in or out, their session is destroyed and rebuilt--so I never have to worry about a user logging out and still showing as being "online" when I query the database. I'm pretty satisfied with it and quite pleased with myself considering I've only known ruby for about 3 weeks now.

The hardest part was trying to figure out why my controller tests were failing anytime a session was rebuilt--I had to rewrite bits of the test session store as a result, but it all works as expected now. I didn't realize at first that a different session storage apparatus was used during testing.

Erik

Stewart wrote the following on 15.10.2006 16:07 :

1. I have my session migration set up and I am using active record to store all sessions

2. when a user logs in successfully the following line of code is executed

session[:user] = current_user

if all of this works like a think it does each session should have the entire user model stored within it.

If that's true what's the easiest way to do say an ID search on that user model based on the global session collection.

CGI::Session::ActiveRecordStore::Session

For example if I wanted to find out if a certain user with a certain ID is logged on.

can I do something like this?

CGI::Session::ActiveRecordStore::Session.user.Find_by_id(id)    No you can't. The ActiveRecord::Base#find* methods generates SQL. As the session is stored serialized in the data column, there's no way you can use SQL to select on the value of one of the session's keys. This is the sole reason for the whole discussion that just took place.

Lionel.

Erik wrote the following on 15.10.2006 02:51 :

  

This is not a major problem, but you must keep in mind that you have two places where your user_id is stored, you'll have to maintain consistency.      Well perhaps, but only if you're not rebuilding your session when you change credentials. I'd love to illustrate:

For instance, I wanted a more secure session store for my application. I didn't like the fact that the default handling used only the session_id for granting access to session data and I didn't like the way most "drop-in" authentication systems didn't rebuild the session during credential changes or access escalations. So I wrote my own. In my own session handling, I use a mysql back end with a schema just like the one I showed you earlier in the thread but it also stores the ip address in a field I call 'host'.

Interesting. I like having the IP in the session (not only for adding another level of security but also for debugging). You could have used the standard session to do that tough, there's no need for a separate column (you get your session by _session_id value as usual and can check for the session[:host] value in your authorize method). Anyway having access to the host value in the table itelf has other advantages than just another security check as I said.

Note : there could be problems with users behind proxy pools, but I'm not sure if this is still relevant (IIRC AOL used proxy pools and you couldn't be sure that the same user wouldn't change proxy during one session). Anyone knows if the problem still occurs?

  I wanted my sessions to use the remote IP address of the client in addition to the session_id for controlling access to the session storage. That would help protect my users from session fixation a little bit, but I also didn't want to just destroy sessions whose session_id didn't have a matching host address. Doing that would still leave my my sessions open to denial of service attacks. For that problem, I had to modify ActionController::CgiRequest so I can non-destructively rebuild sessions. I also did not want to endure the overhead of hitting the database 2 or 3 times extra with each request just to tag each session with the user_id and host, so I also modified CGI::Session and wrote my own session store where you can set the host and user_id in the session object where they will be handled during normal session reads/writes.    This is a neat approach and not so complex. You change the Session interface by adding new methods to your implementation but it isn't a major drawback (you probably won't ever need to go back to another implementation and at the same time can make your implementation evolve to leverage memcache's scalability if you need to).

Lionel.

Theres not really any way to search for an active session by the user ID in any existing "canned" rails session handling scheme--at least not to my knowlege. It might be a very simple alternative for someone to simply add a last_seen column to their users table and find your "online" users by searching that column for users who accessed the site in the last 15 minutes (or whatever duration you prefer). The drawback there is another hit to the database with every request, but that's really not so bad in most cases.

@Lionel: I'll see if I can distill my session hack and post it somewhere, if you would like to see it.

Erik

By doing that, you would have a list of active sessions, yes. However, to find out which registered users were online (as opposed to anonymous users), you would have to iterate the session records and check each one for a user_id. On top of that, if you wanted to pull information about each user that you found logged in (like user name, email address, whatever you might have in the users table), you would have to query the database again for each active session with a user_id--which can quickly add up to a LOT of extra database hits and consequential slow-down.

Erik

I posted on my blog about the session store I am currently using (the one I was describing above). Most of my source code for the session (if not all of it) is on display there.

http://burningtimes.net/articles/2006/10/15/paranoid-rails-session-storage

Erik

Erik wrote the following on 16.10.2006 13:23 :

I posted on my blog about the session store I am currently using (the one I was describing above). Most of my source code for the session (if not all of it) is on display there.

http://burningtimes.net/articles/2006/10/15/paranoid-rails-session-storage

Nice documentation ! I've not read everything yet but this seems well done.

Thanks,

Lionel

Nobody has criticism to offer me?

Erik