looking for feedback - Working on a model in in memory instead of the database

I have an interesting scenario and would love some advice: We're
wondering if it's possible, or even a good idea, to allow a user to
work on a copy of a model in the session or somewhere else and only
save it to the database when the user goes through a 'save' process. We
still want validations to work and we really don't want to duplicate
code on the client if we don't have to.

The basic desire is to allow the user to be able to work on a model,
some related models, etc, do some calculations, and then save the file
when they're done. Could be lots of related models too.

I'm looking for solutions.... session might not be it. Or is this a
bad idea? We're trying to use a 'fat client' model where the user's
work is done "in memory" and saved to the disk when they choose "save".
We're trying to avoid data loss as well as accidental data loss from
the user.

A session might work, depending on how much data you're talking. An
alternative might be my acts_as_draftable plugin. It's a lot like
acts_as_versioned in concept. It saves a draft record in another
table. When you're ready to save and publish the changes, you can
then save the draft over the actual record.

http://svn.techno-weenie.net/projects/plugins/acts_as_draftable/README

Sounds pretty interesting… I will look into this further! Thanks a lot!

FYI… we could be talking about a lot of data. The first thing that came to mind was acts_as_versioned but this could be it.

I'm confused as to how you define 'client'. And without saying what
problem you are trying to solve it's difficult to give an answer to
your question. Whether you use the database or a session, it's still
hitting the file system to save the data. The actual session data
isn't stored in the client if that's what you were thinking. Nothing
wrong with storing temporary data in a session, but once it goes into
a model why use sessions over the database? By the time you run into
performance issues with a database you will have pretty much the same
issue with sessions, if not more. Serializing a model into the
session will also add more overhead.

@snacktime

Thanks for your input. I am aware of the issues you’ve listed and that’s why I posed the question. I am a bit uncomfortable with storing things in session. As for your statement

" Whether you use the database or a session, it’s still hitting the file system to save the data."

That’s not true as we are not using the file system for session storage.

We want the ability for a user to work with and easily discard all changes if desired. Think Excel.

You make changes to the data on the spreadsheet, but you don’t save it unless you hit the save button. That’s the desired outcome. How we get there is still up in the air but the first thing that came to mind was retrieving the model into session and working with it there throughout the various screens on the client.

Type of client is irrelevant. It could be anything… pda, Flash, another program entirely, or plain HTML. I’m not at liberty to discuss the specifics of this project, or I would provide more. Just assume that peformance is not the concern at this point.

Acts_as_draftable might be just what we’re looking for but I invite other suggestions… and I do consider “Don’t do it!!! It’s not worth it!” a valuable suggestion as well.

Thanks again!

Thoughts:
We are considering a similar scenario: Allow a user to create a study
of questions/answer fields - some exist in the database (w/ data) and
some newly created questions/anwers. User enters data on the study
(form).

This could probably be implemented most easily as a dynamically created
database view. But can it be implemented effectively in ROR? If so,
would it look like Brian's description of many models of questions sets
and much logic to order and present the study questions plus controlled
model 'saves'.

Or - if your audience is small (mine is), do you implement as a Ruby
client-server application that deals directly with the updateable
database views. This seems more straightforward and less like we are
trying to fit the application to the technology!

Would appreciate more thoughts.

dan m

If performance isn't an issue, I really don't see why you would not
use the database. I would think it would be much easier to just
maintain the state using a tag of some type then trying to use
sessions. At least that's how it looks given the information you
have provided.

Just thinking out loud here, but what about making a few models that use an sqlite :memory: database for when they are working on the models in memory. ANd then when they click save, you can clone the object over into the permanent disk backed database?

Cheers-

-Ezra

@snacktime:

I may have one model and several child models with belongs_to, etc. How would I allow a user to roll all of those back if I have saved over the original record in the database?

(I would love to just use the database but again, need to cover my bases.

@Ezra
That’s certainly intriguing… how would you do something like that?? (in-memory sqlite db)

-bph

Ok now i see what you are trying to do. Good question. I'm actually
not sure how I might do that. The first thing that comes to mind is
to have two records of the data. One temporary that is being worked
on, and one permanent that is updated when the user elects to finalize
their work. You wouldn't need to create any additional tables, just
an additional field in the appropriate table to keep track of the
state. If you expand on this you could also let users keep different
versions of the same set of data they are working on, revert to past
versions, etc..

mysql supports something called The MEMORY (HEAP) Storage Engine which
does store the table defenitions on file and the data in the memory.
You could use it or your temporary data. and then store it back to the
original, or clear it.

as i see your name in this thread, why not cache the models via backgroundrb? or is that the same thing as saving it in the session?

Brian-

Well you would need to have sqlite3 installed and working and then add this to your database/yml:

sqlite3:

:adapter: sqlite3

:dbfile: ‘:memory:’

Then you make a model that uses that db for its connection. Or you could make an abstract model to inherit from if you need multiple models in the memory db. And then you need to use the Migration code to setup those tables which you would have to research a bit.

class MemoryModel < ActiveRecord::Base

#code here

end

MemoryModel.establish_connection( ‘sqlite3’ )

THats the basic idea. Then when the user wants to commit to the real database you could just create your normal model instances and use update_attributes to update them with the data from the memory db.

Cheers-

-Ezra

I use sqlite all the time and I had no idea you could even do that! That’s awesome! Thanks man!

-bph

Brian Hogan wrote the following on 26.08.2006 06:47 :

I use sqlite all the time and I had no idea you could even do that! That's awesome! Thanks man!

-bph

    Brian-

    Well you would need to have sqlite3 installed and working and then
    add this to your database/yml:

    sqlite3:
      :adapter: sqlite3
      :dbfile: ':memory:'

    Then you make a model that uses that db for its connection. Or you
    could make an abstract model to inherit from if you need multiple
    models in the memory db. And then you need to use the Migration
    code to setup those tables which you would have to research a bit.

    class MemoryModel < ActiveRecord::Base
      #code here
    end

    MemoryModel.establish_connection( 'sqlite3' )

    THats the basic idea. Then when the user wants to commit to the
    real database you could just create your normal model instances
    and use update_attributes to update them with the data from the
    memory db.

Although I like the idea, I guess it doesn't scale very well though : what happens when you need to use several servers to handle the load? Does sqlite3 with memory databases at least allow to share databases between processes on the same server?

If it doesn't work, I believe you are better off adding tables in your existing DB (or your users may not understand while the data they are working on suddenly disappear and reappear later if not worse...).

Lionel.

As far that I know ( I read it in the official mysqldoc) Momery tables
are avaiable in mysql too, not only in sqlite3 . And with Mysql 5 you
would have graet clustering and load balancing options available. So my
guess (I don't have any expirience in large mysql system) is that
scaleabilety shouln't be an issue here.