Empty collection returned for model with has_many relationship

I have two models:

Exercise, and Step. Exercise has many steps.

Basically, when I add a step to a new exercise's "steps" array, it saves, and everything works. If I then get the exercise from the model (via find), clear its array, then add the step again, I cannot save the step to the exercise. Here is a console dump which shows exactly what happens:

bigsley wrote:

I have two models:

Exercise, and Step. Exercise has many steps.

Basically, when I add a step to a new exercise's "steps" array, it saves, and everything works. If I then get the exercise from the model (via find), clear its array, then add the step again, I cannot save the step to the exercise. Here is a console dump which shows exactly what happens:

----------

a = Exercise.new

=> #<Exercise id: nil, name: nil, description: nil, image: nil, created_at: nil, updated_at: nil>

a.name = "The Name"

=> "The Name"

b = Step.new

=> #<Step id: nil, description: nil, picture: nil, number: nil, exercise_id: nil , created_at: nil, updated_at: nil, name: nil>

c = Step.new

=> #<Step id: nil, description: nil, picture: nil, number: nil, exercise_id: nil , created_at: nil, updated_at: nil, name: nil>

a.save

=> true

b.save

=> true

c.save

=> true

a.steps << b

=> [#<Step id: 35, description: nil, picture: nil, number: nil, exercise_id: 26, created_at: "2009-01-06 02:26:36", updated_at: "2009-01-06 02:26:38", name: nil

]

a.steps << c

=> [#<Step id: 35, description: nil, picture: nil, number: nil, exercise_id: 26, created_at: "2009-01-06 02:26:36", updated_at: "2009-01-06 02:26:38", name: nil

, #<Step id: 36, description: nil, picture: nil, number: nil, exercise_id: 26,

created_at: "2009-01-06 02:26:36", updated_at: "2009-01-06 02:26:40", name: nil> ]

a.save

=> true

a = Exercise.find_by_name("The Name")

=> #<Exercise id: 26, name: "The Name", description: nil, image: nil, created_at : "2009-01-06 02:26:35", updated_at: "2009-01-06 02:26:35">

a.steps

=> [#<Step id: 35, description: nil, picture: nil, number: nil, exercise_id: 26, created_at: "2009-01-06 02:26:36", updated_at: "2009-01-06 02:26:38", name: nil

, #<Step id: 36, description: nil, picture: nil, number: nil, exercise_id: 26,

created_at: "2009-01-06 02:26:36", updated_at: "2009-01-06 02:26:40", name: nil> ]

a.steps.clear()

=>

a.steps << b

=> [#<Step id: 35, description: nil, picture: nil, number: nil, exercise_id: 26, created_at: "2009-01-06 02:26:36", updated_at: "2009-01-06 02:26:38", name: nil

]

a.steps << c

=> [#<Step id: 35, description: nil, picture: nil, number: nil, exercise_id: 26, created_at: "2009-01-06 02:26:36", updated_at: "2009-01-06 02:26:38", name: nil

, #<Step id: 36, description: nil, picture: nil, number: nil, exercise_id: 26,

created_at: "2009-01-06 02:26:36", updated_at: "2009-01-06 02:26:40", name: nil> ]

a.save

=> true

b.save

=> true

c.save

=> true

a = Exercise.find_by_name("The Name")

=> #<Exercise id: 26, name: "The Name", description: nil, image: nil, created_at : "2009-01-06 02:26:35", updated_at: "2009-01-06 02:26:35">

a.steps

=>

----------

What is going on here? This is so anti-intuitive - how could a bug like this exist and not have been detected hundreds of times before?

Yes, I would regard this as a Rails bug. The clear method is not setting the foreign key to nil on the in-memory object.

It's only being manifest though because of the unnecessary saves you're doing, meaning that the problem has probably been rarely seen.

Mark Reginald James wrote:

I take this back:

It's only being manifest though because of the unnecessary saves you're doing, meaning that the problem has probably been rarely seen.

The problem has been exposed by the new ActiveRecord change detection code.

----------

What is going on here? This is so anti-intuitive - how could a bug like this exist and not have been detected hundreds of times before?

Yes, I would regard this as a Rails bug. The clear method is not setting the foreign key to nil on the in-memory object.

Even if it were setting it on the in memory object it wouldn't be in
memory object you think it is. Because a was reloaded, the objects in
a.steps are not the objects b & c (although they do correspond to the
same database rows).

Fred

Frederick Cheung wrote:

Even if it were setting it on the in memory object it wouldn't be in memory object you think it is. Because a was reloaded, the objects in a.steps are not the objects b & c (although they do correspond to the same database rows).

The problem happens before a is reloaded.

I just did some experiments with puzzling results. Would you be able to help explain them Fred?

Loading development environment (Rails 2.2.2) >> a = Exercise.find(1) => #<Exercise id: 1> >> b = Step.new => #<Step id: nil, exercise_id: nil> >> b.save => true >> c = Step.new => #<Step id: nil, exercise_id: nil> >> c.save => true >> a.steps << b => [#<Step id: 19, exercise_id: 1>] >> a.steps << c => [#<Step id: 19, exercise_id: 1>, #<Step id: 20, exercise_id: 1>] >> b.object_id => 70254505869500 >> a.steps.first.object_id => 70254505813160 >> c.object_id => 70254505838840 >> a.steps.last_object_id => 70254505838840

Why does only the first object in the collection have a different object_id, and why does this difference go away if the empty a.steps is loaded before b is added? Is it a proxy thing?

I added a few lines to the has_many code to set the foreign keys on the in-memory records to nil. But because of the above behaviour the fk doesn't change on the local variable that held the first element added to the collection, unless the collection had been pre-loaded.

Mark Reginald James wrote:

Frederick Cheung wrote:

Even if it were setting it on the in memory object it wouldn't be in
memory object you think it is. Because a was reloaded, the objects in
a.steps are not the objects b & c (although they do correspond to the
same database rows).

The problem happens before a is reloaded.

I just did some experiments with puzzling results. Would you be able to help explain them Fred?

Loading development environment (Rails 2.2.2) >> a = Exercise.find(1) => #<Exercise id: 1> >> b = Step.new => #<Step id: nil, exercise_id: nil> >> b.save => true >> c = Step.new => #<Step id: nil, exercise_id: nil> >> c.save => true >> a.steps << b => [#<Step id: 19, exercise_id: 1>] >> a.steps << c => [#<Step id: 19, exercise_id: 1>, #<Step id: 20, exercise_id: 1>] >> b.object_id => 70254505869500 >> a.steps.first.object_id => 70254505813160 >> c.object_id => 70254505838840 >> a.steps.last_object_id => 70254505838840

Just a WAG, but a.steps.first.object_id isn't the same reference method as a.steps.last_object_id

Is the first.object_id creating a new reference to that object, whereas last_object_id is returning the reference present in a.steps?

Try the experiment again with the same object referencing method used for both first and last.

Because .first will hit the database and load a fresh copy (if the collection is not already loaded). I waffled about this a bit a while ago: First, foremost and [0] - Space Vatican

Fred

So is this a known bug? Is this fixed in 2.2.2? I'm currently running rails 2.1.1.

Frederick Cheung wrote:

Why does only the first object in the collection have a different object_id, and why does this difference go away if the empty a.steps is loaded before b is added? Is it a proxy thing?

Because .first will hit the database and load a fresh copy (if the collection is not already loaded). I waffled about this a bit a while ago: First, foremost and [0] - Space Vatican

Thanks Fred for pointing out the new behaviour of "first" and "last".

However that's not the problem here. You get the same behaviour if you use steps[0].

Looking at it again after some sleep, I see that the different b object is actually loaded at the "a.steps << b" line. That is, the object is added by updating the foreign key in the database, loading the collection, then folding in any new_records, which ends up giving you the different b from the database.

The association_collection << method could easily be changed to respect the identity of objects being added to a collection. Should this be done to respect the POLS, violated for the OP?

The association_collection << method could easily be changed to respect the identity of objects being added to a collection. Should this be done to respect the POLS, violated for the OP?

Possibly. This area is fraught with subtleties though.

Fred

bigsley wrote:

So is this a known bug? Is this fixed in 2.2.2? I'm currently running rails 2.1.1.

The issue is present in 2.2.2, and probably edge. I'm looking into a fix, but as Fred says, there are several subtleties. I suggest you watch this ticket:

http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/1706

Assuming your console log was an attempt to investigate a problem with some real app code, you should be able to work around the problem by eliminating some unnecessary saves and some object reuse. You can post your real code if you like.