included model attribute selection using :symbol[]

Ticket 7147, among others, discusses support for restricting
attribute selection for finds that do eager-loading.

A core team member was not enthusiastic about approaches
that required the parsing of :select option SQL, so I've
tried another approach in an attempt to get core team
support for this capability.

The details are at

   http://dev.rubyonrails.org/ticket/7147#comment:12

but the key aspect is the use of the Symbol [] method
to specify a list of attributes to be selected for each
eager-loaded model: e.g.

  :include => {:books[:title, :stock, :price] => :author}

I've tried to minimize the impact of defining a new method
on a central class by forcing use of :symbol[] in any other
context to either return a

   NoMethodError: undefined method `[]' for :symbol:Symbol

or to make use of an already-defined Symbol [] method.

Are their other dangers with such a definition that makes
you uncomfortable?

Are their other dangers with such a definition that makes
you uncomfortable?

I just don't think that the feature justifies adding such strange and
counter-intuitive syntax. I'm all for adding methods to core classes
that aid developers, every day. Things like 1.day.ago etc but just
for this one case it's not justified.

Michael Koziarski wrote:

Are their other dangers with such a definition that makes
you uncomfortable?

I just don't think that the feature justifies adding such strange and
counter-intuitive syntax. I'm all for adding methods to core classes
that aid developers, every day. Things like 1.day.ago etc but just
for this one case it's not justified.

In my view the :assoc[:attr1, :attr2] is the most intuitive
way of selecting attributes for for eager-loaded associations,
naturally extending the existing syntax.

The alternatives are:

1. Parsing the :select option,
2. Not providing the ability to restrict attribute selection, or
3. A less-streamlined but more convention syntax that has the same
    effect: e.g. Adding an ActiveRecord class method "eager_select":

:include => {eager_select(:books, :title, :stock, :price) => :author}

1. Parsing the :select option,
2. Not providing the ability to restrict attribute selection, or
3. A less-streamlined but more convention syntax that has the same
    effect: e.g. Adding an ActiveRecord class method "eager_select":

I'm personally not sold on this particular optimisation technique. I
think the real problem here is that it's still too hard to load a
graph of objects out of a find_by_sql call. If that were fixed you'd
be able to optimise cases like this, without too much complication to
AR's codebase.

Michael Koziarski wrote:

I'm personally not sold on this particular optimisation technique. I
think the real problem here is that it's still too hard to load a
graph of objects out of a find_by_sql call. If that were fixed you'd
be able to optimise cases like this, without too much complication to
AR's codebase.

Michael, would you be able to expand on this?

He means there’s no way to do custom eager loading. Namely, a find_by_sql (or any find call without :include for that matter) only ever instantiates models of a single type. There is a much larger class of performance problems that can be solved by allowing instantiation of arbitrary models out of any given query (ie. model graphs based on dev-supplied JOINs).

Personally I believe this should be possible not only with find_by_sql, but also find + :joins/:select, since that allows the developer to continue leveraging the power of AR query generation which I rely upon heavily in my current project.

I’m working on a solution to this, but I have quite a few deadlines to get through before I can devote serious energy to it. The hard part is figuring out what the interface should look like.

Gabe da Silveira wrote:

He means there's no way to do custom eager loading. Namely, a find_by_sql (or any find call without :include for that matter) only ever instantiates models of a single type. There is a much larger class of performance problems that can be solved by allowing instantiation of arbitrary models out of any given query (ie. model graphs based on dev-supplied JOINs).

Personally I believe this should be possible not only with find_by_sql, but also find + :joins/:select, since that allows the developer to continue leveraging the power of AR query generation which I rely upon heavily in my current project.

I'm working on a solution to this, but I have quite a few deadlines to get through before I can devote serious energy to it. The hard part is figuring out what the interface should look like.

Gabe, are you talking about eager-loading finds having custom joins
that don't just load along a chain but match-up arbitrary foreign keys?
I and others have written patches for find_by_sql/:finder_sql
that allows them to accept an :include option, instantiating the
object hierarchy from SQL that uses field aliasing of the Rails eager
loading form:

http://www.kellogg-assoc.com/articles/2006/11/05/eager-finder-sql
http://mrj.bpa.nu/eager_custom_sql_rails_1.2.rb

Or are you talking about having eager-loading instantiate the
cross and back links of the object hierarchy? A patch that
implemented the first step of this -- matching and caching not just
the instances seen in each eager-loaded association, but those in
each model class -- was posted here several weeks ago by Fred Cheung.

A simple eager-loaded object tree is the most common query
required in a Web app, and being able to restrict the attributes
selected can make a huge speed and space difference when unwanted
fields are large and the eager load runs deep.

Gabe, are you talking about eager-loading finds having custom joins
that don't just load along a chain but match-up arbitrary foreign keys?
I and others have written patches for find_by_sql/:finder_sql
that allows them to accept an :include option, instantiating the
object hierarchy from SQL that uses field aliasing of the Rails eager
loading form:

http://www.kellogg-assoc.com/articles/2006/11/05/eager-finder-sql
http://mrj.bpa.nu/eager_custom_sql_rails_1.2.rb

I think that the feature you've implemented there should probably be
rolled in to active record at some point, whether or not that's for
2.0 I'm not sure.

However this plugin clearly minimises the need for the custom
:symbol[] approach, or is there another case I haven't thought of?

> Or are you talking about having eager-loading instantiate the

cross and back links of the object hierarchy? A patch that
implemented the first step of this -- matching and caching not just
the instances seen in each eager-loaded association, but those in
each model class -- was posted here several weeks ago by Fred Cheung.

A simple eager-loaded object tree is the most common query
required in a Web app, and being able to restrict the attributes
selected can make a huge speed and space difference when unwanted
fields are large and the eager load runs deep.

Full bi-directional associations are probably too much to push out for
2.0, but in the long run it'd be nice to get them in there.

If finder_sql + :include can cover all the cases here, I think we'd be
better off looking to merge that functionality into rails, rather than
building some more syntax for a specific place we need it.