how to manage states in a CRUD way?

Hey all,

I have a Job model and the job can be opened, closed and fulfilled. It can be re-open, re-closed and "re-fulfilled" several times and I want to keep track everytime this happens (by keeping the date of each change). What's the best way to handle this situation in a CRUD/Restful way?

thanx in advance

Pat

Patrick -

ok thanx for the info and yes I want them :slight_smile:

did I miss something or is there any table I should create to make it work?

Patrick -

oh, and out of the box the plugin doesn't track the history of state changes.

But you could likely use it in combination with something like acts_as_audited:

http://agilewebdevelopment.com/plugins/acts_as_audited

Jodi

I want them!!! I've got many others, which I'm trying to get into shape to contribute back...

Some things I've added:

* State properties: each state can have a hash to store whatever you want to store (a caption, a color, granular access control restriction. * Event properties: similar to State properties, I use :caption, :public (to specify whether it should be shown or if it's triggered by some backend stuff), :confirm (same as link_to), :position (to make the public ones appear in the same order every time). * Composite states: I can say e.g. "my_model.an_assoc.find_all_in_state :active" and it searches "WHERE state IN ('posted', 'selected', 'contacted')" (it's even recursive so I can have a tree of states). This is what's referred in UML as non- concurrent composite states (concurrent states doesn't make sense in this context). * Queries: some query methods to find out leaf states (in the compsite tree), root states, possible events given the current states (to automatically show those that can be triggered to the user), a helper for select_tag usage, paginate_in_state (an ugly hack, but useful), and some others.

  Sorry for the off-topic!

nachokb

Looks nice!

Have you seen http://agilewebdevelopment.com/plugins/acts_as_trackable ?

Thanks!

nachokb

Jodi and nachokb, please keep this topic alive, Is there a blog or repo where we can pull these enhancements?

Just a quick question, how do you manage field validation when the state changes? I tried with the :guard proc but it didnt catch the erroneous field at the right time. It was a long time ago so I dont remember the specifics but I am curious to know how others are dealing with it.

I can send you Jodi's patch if you want and if he didn't already send it to you (I'm sure he would agree right? :slight_smile:

please do Patrick.

I'm away for the Canadian long weekend - would like to resume our
thoughts on a combined patch next week.

Have a good week.

Cheers, Jodi

Right now, we are both really busy. We are planning on releasing something, though. My code is really messy so I ask you to wait a little.

Thanks!

nachokb

This is my take on state machines... I would advise you to first take a piece of paper, draw a state diagram (similar to the UML SD). Each node is a state, call it pereferably with an adjective ("closed"). Each state con have many outbound and inbound transitions. Those without outbound are called "final" states; only one should have no inbound, and it will be called the "initial" state (you must specify that to aasm). Now, for each and every transition you must determine the "action" or "event" that triggers it (just a name, preferably a verb, like "close"). Aside from that, you should also think about some conditions that should be met for that transition to take effect (in aasm, it's close your model, for example. You could also check that current_user == my_model.author, my_model.some_field_not_empty?, etc. Furthermore, you also have a few other hooks, but this time not for validation (as they can't stop a transition from happening), but mainly for side-effects. AASM provides you with hooks for :entering a state, another one for after a state has been :entered, and a final one for when you are :leaving a state. In your case, I guess you are trying to use AR's validators together with these. I know something like this works:

validate_uniqueness_of :some_unique_field, :if => &:closed? # Validates it when it is about to save the newly state-changed model.

I want to split :guard into :precondition and :postcondition (and warn the user that :precondition s should never ever cause side-effects). This way I can enumerate those possible actions that meet their precondition to the user. Suppose this scenario:

  * Job model has states :active, :closed.   * 1 transition :close (from :active, to :closed).       * that transition has a precondition: that the user is an admin       * and a postcondition: that the user filled in the hours it took her to finish it.   * :active, on :leaving, just logs "#{model.name} is no more :active".   * :closed, on :entering, just logs (let's keep it simple) "#{model.name} is now :closed".   * I never found a case for :entered...

I can have, in the view, an enumeration of the possible events given the current state (let's assume :active). In this case, the only event is :close. Further, I can select only those events which meet their precondition. If the user is an admin, my app would show to her the "Close" button. Otherwise, it wouldn't. Now suppose I haven't separated pre- and post-conditions for a moment... if I filter the events to those meeting its "guard", the Close button would never show, even if the user is an admin, because she hasn't yet filled the hours.

Furthermore I don't find specially useful the :entering, :entered and :leaving hooks, I would prefer to have one (only one) attached to the transition. Keep in mind that more than one transition can lead to the same state (or leave it). That one hook would know the previous state, its parent transition, and of course the target state...

Bye!

nachokb

The main idea behind something like this is to model the state flow in a declarative way (building our own DSL). That way, it's far, far easier to read and maintain. I've actually shown it to an analyst and understood it right away. That's the beauty of Ruby... I've got so much behaviour expressed in so little code, it's almost a "dump" of a state diagram (though there are some things I can't connect without custom code *). I've wanted to do something similar to railroad.rubyforge.net taking advantage of this dsl...

nachokb

* in a project I've got 3 models acts_as_state_machine. Some transitions are what I called "public" (meaning that the user sees a button and triggers it), but some other don't show to the user. Those are triggered on transitions in other (related via AR) models. For example, once a job_application has its :hire event triggered, its related person gets its :hire also triggered, and all the **other** job_applications of that person get its :close triggered. I represent those with dashed arrows between each model's state diagram, but can't pick that up automatically (without inventing some really complex dsl which would let me do "all my person's job_application, **except** me").

A question to everyone out there: would you manage a model's state as a nested singleton resource??

I'm a little late here, but you might want to take a look at the plugin has_states: http://svn.pluginaweek.org/trunk/plugins/active_record/has/has_states/ . Documentation is a bit lacking right now, which I'm working on, so you might want to take a look at the unit tests for usage.

It's based on the acts_as_state_machine plugin, but a lot of it is rewritten. A brief overview of the differences: * Events/States stored in database tables (events/states, respectively) * Event callbacks (before_*, *, and after_* for event, e.g. before_execute, execute, after_execute) * Passing args to events (e.g. execute!(1, 2, 3)) * State callbacks (before_enter_*, before_exit_*, after_enter_*, after_exit_*) * Inheritance support * Tracking state changes (optional, can be turned off on a per-model class) * Dynamic initial states (e.g. has_states :initial => Proc.new {...}) * Callbacks canceling the event * Module extension for has_many associations (e.g. highway.cars.* where * is a state, such as highway.cars.idle, highway.cars.parked) * Support for state deadlines through has_state_deadlines plugin (i.e. time at which a state must change or it'll automatically change) * > 400 unit tests

Anyway, thought I'd mention it .. good luck!

* > 400 unit tests

WOOOHHOOOOO you are the man! I promise to take a look later...

* Events/States stored in database tables

Yeah, I did that in another project (a Java project), and didn't want to "hardcode" the states, having them on a db (actually, I had an xml for other reason) would allow me to change that at runtime... but I found that too messy and pretty much unnecessary. Later, I was thinking in separating "parameterization" from "configuration", so my xml would never change once it reached stability, and if they wanted to change e.g. which role could do it, then the xml would reference a value set in a configuration file (a properties file). I never fully implemented that. Then I came to Ruby, and aasm particularly, and really liked how concise it was (even if it *is* hardcoded, it's really part of the model, and it allows you to express the state flow in a natural manner). Would you care to explain the rationale behind moving the parameterization into the db?

* Passing args to events (e.g. execute!(1, 2, 3))

In my rethinking about these things, I came to think that it would be useful, but never found a single example where I would use it. I think of it like they take all they need from their context... did you find it useful?

* Inheritance support

I guess that's what I called "composite states". Is it? I mean, a state can have substates, and you could declare something (say, a transition or a property) in the parent which all its children would have (I never implemented that). Plus, you can query for a superstate, say :active, an get all the models which are in :published or :another_one, but not in :closed...

* Tracking state changes (optional, can be turned off on a per-model class)

That's good. I've been doing it separately (was about to turn it into a plugin, or reuse acts_as_trackable). Do you have another model, say "Event", as a log?

* Dynamic initial states (e.g. has_states :initial => Proc.new {...})

Did you find an scenario which would use it? I can't imagine right now...

* Callbacks canceling the event

What's the difference with :guard ? (or my :precondition and :postcondition)

* Module extension for has_many associations (e.g. highway.cars.* where * is a state, such as highway.cars.idle, highway.cars.parked)

I implemented that, only to find later that you already have that for free!!! Every ClassMethod acts_as_state_machine gives you can be accessed (appropriately scoped!) from within the association proxy!!. So you can do highway.cars.find_in_state :active....

* Support for state deadlines through has_state_deadlines plugin (i.e. time at which a state must change or it'll automatically change)

I'm interested! How do you do that? got a cron hook?

* > 400 unit tests

Again, GREAT! I feel miserable...

On a sidenote: how do you reference callbacks from within the db? a method name? Losing blocks was the deciding factor in my acceptance of aasm's parameterization from a class... Perhaps we could later explore some alternatives, like separating the state flow declarations from the model (somewhat like an observer).

Good work!

nachokb

manner). Would you care to explain the rationale behind moving the parameterization into the db?

There are a few reasons behind this. First, and foremost, was related to ensuring that the database is normalized. Since the plugin stores state changes, state deadlines, etc. I was uncomfortable with using the state name in these state_change/state_deadline records. Instead, I wanted those records to reference states stored in the database. This also helps with simplifying queries and defining associations. In addition, that state is also referenced in the model that has the state, so again we're normalizing the database quite a bit.

In addition to this, I took into consideration the fact that, as your project progresses, states may get removed and become "inactive", i.e. they are no longer a part of the state machine. When this occurs, the states should still be valid in any state_change/state_deadline/model that references it, but it should no longer be declared in your model's code. By storing it in the database and forcing the developer to define which states are active in the model, we can continue to have associations working between these various models on inactive states and still ensure referential integrity. If it were only defined in the class, it would be difficult to track all of the states/ events that have been used over time and were, at one point or another, valid. You would have to look through all the records in your database and find names of events/states that were used at one point. It doesn't smell right to me.

There may be arguments both ways, but this is what I felt I was most comfortable with.

In my rethinking about these things, I came to think that it would be useful, but never found a single example where I would use it. I think of it like they take all they need from their context... did you find it useful?

I have found it very useful. I often want to perform an operation when an event occurs, based on certain data that is being passed in. For example, I had an InvoiceTransaction model that had an event call capture. Capture took a single value, which would basically make a partial charge on a payment account through a payment gateway. By passing in the value, I was able to do several things:

1. Define a guard which determines whether the invoice transaction goes to completed (meaning no more captures can be made) or captured (meaning more captures can be made) based on whether the value being captured has reached the maximum total amount that can be captured. 2. Define an on-invocation callback that makes the call to the payment gateway

e.g.

class InvoiceTransaction < ActiveRecord::Base   event :capture do     transition_to :completed, :if => Proc.new {|t, value| t.max_capture_value == (t.total_captured_so_far + value)}     transition_to :captured   end

  def capture(value)     # send to gateway     true   end end

transaction.capture(50.00)

That's obviously not the exact code I use, but it's one use I found for passing in parameters.

I guess that's what I called "composite states". Is it? I mean, a state can have substates, and you could declare something (say, a transition or a property) in the parent which all its children would have (I never implemented that). Plus, you can query for a superstate, say :active, an get all the models which are in :published or :another_one, but not in :closed...

Hmm, that's an interesting idea, but I was actually referring to the fact that states, events, and transitions defined in a class are only valid and accessible by that class and all its subclasses. I believe there are a few issues with how transitions are handled in subclasses in acts_as_state_machine, though I could be wrong.

That's good. I've been doing it separately (was about to turn it into a plugin, or reuse acts_as_trackable). Do you have another model, say "Event", as a log?

There are three models in has_states: State, Event, and StateChange. StateChange tracks the from_state, to_state, event, and when it occurred. For the initial state change (the first state a model is set to), the from_state and event are set to nil. Otherwise, all fields are always set. In addition, it tracks all changes, regardless of whether that same change has already occurred (e.g. loopbacks). When accessing a state change, you can choose whether to access the first, last, or all of the changes. For example, car.parked_at(:first), car.parked_at(:last), and car.parked_at(:all).

> * Dynamic initial states (e.g. has_states :initial => Proc.new {...})

Did you find an scenario which would use it? I can't imagine right now...

There is a scenario I'm using it for, although it may be invalid since it may call for inheritance where the subclass overrides the superclass's initial state, but this is something I have to re- evaluate. The scenario involves an InvoiceTransaction which starts off either as a payment to an individual from a company or from an individual to a company. This may simply be a flaw in-of-itself, where there is a need for a class hierarchy that separates this functionality. I decided to keep it in there for now since I already had it implemented and thought I would let the plugin be used throughout various projects and see if I ever find another need for it. If it seems that no more scenarios come up, I'll axe it out of the implementation.

What's the difference with :guard ? (or my :precondition and :postcondition)

Callbacks canceling events are meant to parallel the structure used in callbacks for ActiveRecord. For example, you would normally cancel a save through the use of a validation. However, if any callback (e.g. before_save, after_save, etc.) explicitly returns false, this will cancel the save as well. As a result, in order to make it familiar to ActiveRecord callbacks, state/event callbacks act the same way. Currently, the scenario in which I make use of this has to do with making changes to other models as a result of an event being invoked. For example, using InvoiceTransaction as an example, I have guards setup to determine whether the model transitions to :completed which invokes a web service to actually send the capture to the payment gateway. If that fails, I want to cancel the event. This really couldn't be done as a guard, because you only want to talk to the gateway if all other guards have passed. You could possibly hack this by making sure it's the last guard, but I don't personally find that as clear.

> * Support for state deadlines through has_state_deadlines plugin (i.e. > time at which a state must change or it'll automatically change)

I'm interested! How do you do that? got a cron hook?

This is currently implemented via two methods. First, whenever the record is accessed, the deadline for its current state is checked and the deadline event invoked if it has been passed. This ensures that whenever you access the record, it is always in the correct state. Second, we have a backgroundrb worker checking the state deadlines every few minutes and updating the appropriate records. That's probably not the best way to accomplish it, and I haven't personally put much thought into better methods, but it does the job for now.

Anyway, I hope this answers your questions!

OK so you have the diagram declared both in the db and the model? Wouldn't you want to move it out of the model completely (DRYing it up a little?). I implemented something similar before in Java, and had it declared outside the model (it was in xml, but could have been a db). What I had was the model declaring just _the name of the state machine_. So I had many state machines in the xml, each having its out states, events, etc (argh, just remembering the use of the reflection api to specify the conditions makes me vomit). In my current app I've got three models with aasm, and they all happen to have an :active state and a :discarded state (the first is a composite, whose components are different in the different models; the latter one is a leaf state, but each model has different inbound transitions for it) for instance. Do you also have a reference to the model type in the states table? (so I could have a :closed for CurriculumVitae and another for Job and JobApplication). Did you put your models withing plugin? I had troubles doing that :(. Had to copy the model to my app/models. I didn't get your point about zombie states... (after a change in the state machine). The thing about parameterized events is that I always managed to implement what I was trying to do in some other way (and liked that more). I assume in your example that :captured means "partially completed" (if you will). I would simply have (I'll use the aasm idioms, which I've seen you slightly modified).

def capture(amount)   self.pay(amount) # increment the "paid" counter and raises an exception if it needs so (e.g. paid+amount > total).   self.capture! # triggers the event, without parameters end

and the transitions would be the same... the guard, of course, has access to the transition. So why does it need "amount" to begin with?

Hmm, that's an interesting idea, but I was actually referring to the fact that states, events, and transitions defined in a class are only valid and accessible by that class and all its subclasses.

Oops sorry, I don't use STI (if I need something like that, I create modules; it led me to much better naming which just smells good)...

There is a scenario I'm using [dynamic initial states] for, although it may be invalid since it may call for inheritance where the subclass overrides the superclass's initial state, but this is something I have to re- evaluate.

I will always advocate for composition over inheritance.

it. If it seems that no more scenarios come up, I'll axe it out of the implementation.

That's not my intention. I'm trying to find uses I haven't thought of, a clearer view of another perspective.

Callbacks canceling events are meant to parallel the structure used in callbacks for ActiveRecord.

Now I see that what you mean with "Callbacks" is not what I though (I guess your "capture" is a callback; some magic must be involved in order to wrap state-changing behaviour to that method). You are having problems with side-effects in guards/callbacks. Some time ago I made a choice (when I had already implemented my java version and before coming to ruby, so it never made it into any implementation, sadly). I would have "preconditions" (can stop the event, must not have side effects, determine whether a user can even "see" an event), "postconditions" (can stop the event, can have side effects) and "filter" (cannot stop the event, should have side effects). Your web-service example would be a "postcondition". Something I opted to not consider was atomicity: if one "action" with side effect is followed by another, and the last one fails, it's up to my client to rollback the first one (once I had to consume TWO web- services, notifying something; later, they could notify an error and the other one was required to be "unnotified" if it was previously notified... kinds of an extreme case I ended up managing from other place). Making the separation I did (pre, post, filter) is useful in this: only "postconditions" are dangerous wrt to atomicity.

(...) This really couldn't be done as a guard, because you only want to talk to the gateway if all other guards have passed. You could possibly hack this by making sure it's the last guard, but I don't personally find that as clear.

Really? what about

transition :from => :foo, :to => :bar, :postcondition => lambda { | record> record.still_not_complete? && record.notify! }

Please notice that :still_not_complete? does not have side effects, but :notify! does have. They will always be executed in that order, and thanks to short-circuit boolean evaluation, if the first fails, notify! will never be executed...

This is currently implemented via two methods.

Both are awesome! (your side-effects callbacks get triggered when they need to be regardless of access, plus you never get an inconsistent record, perfect). Perhaps we could try to simplify this, but I don't see how... someone?

OK, now, I really, really promise to look into your plugin's, and make more educated comments the next time :slight_smile:

Thanks!

nachokb