I have been trying to access session data (I am using the
ActiveRecordStore option) from a background process (Workling in my
case - but I suspect it should be the same for others). It seems to me
it should be fairly obvious since I can pass the session_id to the
background and the session data is in the DB anyway. But I can't make
it to work.
I looked around and saw a note on this group from Fred:
"you'll need to instantiate the appropriate subclass of
ActionController::SessionStore" but I don't understand this.
How about just passing in the session data hash when you call the
particular workling ob method, something like:
### in ./app/workers/foo_worker.rb
class FooWorker < Workling::Base
def bar(options)
session_data = options[:session_data])
# do something with session.data hash ...
end
end
### in some_meth in ./app/controllers/some_controller.rb
...
FooWorker.asynch_bar(:session_data=>session.data)
...
Just one thing though: if I do this, does it mean I am taking the
session data "out of" ActiveRecord? I was thinking that by only
passing the session ID and by letting the worker retrieve the record
in the DB, consistency would be guaranteed by ActiveRecord. But maybe
I am getting this wrong?
Indeed, I need to do quite a lot of things on this session data in the
background: several workers are going to work on it at the same time,
and one of them will have to create some threads… I am struggling to
understand whether there will be concurrency risks - therefore I
thought I would try and rely on ActiveRecord as much as possible.
Here is in pseudo-code what I had in mind:
### in ./app/workers/foo_worker.rb
class FooWorker < Workling::Base
def bar(options)
session = Session.find(options[:session_id])
# do something with session.data hash ...
end
end
### in ./app/workers/foo_worker2.rb
class FooWorker2 < Workling::Base
def bar(options)
for i in 1..10
Thread.new(options[:session_id]) { |session_id| retrieve
session data and do stuff }
end
end
end
### in some_meth in ./app/controllers/some_controller.rb
...
FooWorker.asynch_bar(:session_id =>session.id)
FooWorker2.asynch_bar(:session_id =>session.id)
...
A couple of problems that jump out regarding accessing/using session
data in that strategy you outlined:
- the background threads/processes might end up using invalid session
data if the session data changes during the time from when you pass in
that session_id and when some background thread/process fetches that
session data from the db.
- you're making multiple round-trips to the db by having each of the
background threads/processes fetch the session data from the db when
they could just access the data by having it passed in at the time
they are called/launched.
A safer/more-robust alternative would be to just pass in a copy of the
user's session data to each of the background workers to work with:
...
### in ./app/workers/foo_worker.rb
class FooWorker < Workling::Base
def bar(options)
session_data = Session.find(options[:session_data])
# do something with session_data hash ...
end
...
end
### in ./app/workers/biz_worker.rb
class BizWorker < Workling::Base
def baz(options)
session_data = Session.find(options[:session_data])
# do something with session_data hash ...
end
...
end
### in some_meth in ./app/controllers/some_controller.rb
...
sd = session.data
FooWorker.asynch_bar(:session_data =>sd.merge({}))
BizWorker.asynch_baz(:session_data =>sd.merge({}))
...
in the case of active record store you can do
ActiveRecord::SessionStore::Session.find_by_session_id( session_id).data
If you wanted a version that would work no matter what the session
store then you probably want to follow the trail of
ActionController::AbstractStore
Thing is… I ran some tests and concurrency is in fact a big problem:
my background workers all need to read and write from/to the session
data. I was thinking that using a lock would help (each process would
have to wait for the other to release the data):
but I am in fact getting an ActiveRecord::ConnectionTimeoutError (and
yet I am trying to release the session as quickly as possible in each
process).
So I was wondering: is there a recommended design to get several
workers to pass info between each other? Say for example:
- a bunch of workers A do a simple task
- a daemon worker B is "listening"
- each time a process A returns a value, it "sends" it to B
- B accumulates these values
I thought that Workling return store would do the trick but it
doesn't: each time you "set" a value it erases the previous one. I
guess each A could store their return value as a row in a DB - but it
seems to me like an overkill?
For some reason I struggle to find info on this on the web.
Not sure what you are attempting to accomplish in those background
threads/processes, or what the reasoning is behind even using multiple
background threads/processes in the first place in light of what you
seem to be trying to do, but the basic strategy you're attempting to
pursue where you have multiple concurrent threads/processes all
reading from and writing back to some user's session data is just
asking for trouble on a number of levels: overwriting of data by one
of the background threads/processes or the user; resource contention
and deadlock; .....
Even if you didn't use the user's session data to work with / store
such data, if you really want/need to pursue this multi-thread/-
process strategy for processing some common data, then you better
design that processing to deal for concurrency issues, which is not a
simple task.
Other alternatives include bailing entirely on the background threads/
processes and just perform the data processing sequentially and reset
the final val(s?) in the user's session once processing is completed,
or you could pass off such sequential processing to be done in some
background process that performs that data processing sequentially in
that background process and persists the final results somewhere else
to be picked up later on and made available to the user in some future
request.