included model attribute selection using :symbol[]

rubyonrails-core
1

Ticket 7147, among others, discusses support for restricting attribute selection for finds that do eager-loading.

A core team member was not enthusiastic about approaches that required the parsing of :select option SQL, so I've tried another approach in an attempt to get core team support for this capability.

The details are at

   http://dev.rubyonrails.org/ticket/7147#comment:12

but the key aspect is the use of the Symbol method to specify a list of attributes to be selected for each eager-loaded model: e.g.

  :include => {:books[:title, :stock, :price] => :author}

I've tried to minimize the impact of defining a new method on a central class by forcing use of :symbol in any other context to either return a

   NoMethodError: undefined method `' for :symbol:Symbol

or to make use of an already-defined Symbol method.

Are their other dangers with such a definition that makes you uncomfortable?

2

Are their other dangers with such a definition that makes you uncomfortable?

I just don't think that the feature justifies adding such strange and counter-intuitive syntax. I'm all for adding methods to core classes that aid developers, every day. Things like 1.day.ago etc but just for this one case it's not justified.

3

Michael Koziarski wrote:

Are their other dangers with such a definition that makes you uncomfortable?

I just don't think that the feature justifies adding such strange and counter-intuitive syntax. I'm all for adding methods to core classes that aid developers, every day. Things like 1.day.ago etc but just for this one case it's not justified.

In my view the :assoc[:attr1, :attr2] is the most intuitive way of selecting attributes for for eager-loaded associations, naturally extending the existing syntax.

The alternatives are:

1. Parsing the :select option, 2. Not providing the ability to restrict attribute selection, or 3. A less-streamlined but more convention syntax that has the same     effect: e.g. Adding an ActiveRecord class method "eager_select":

:include => {eager_select(:books, :title, :stock, :price) => :author}

4

1. Parsing the :select option, 2. Not providing the ability to restrict attribute selection, or 3. A less-streamlined but more convention syntax that has the same     effect: e.g. Adding an ActiveRecord class method "eager_select":

I'm personally not sold on this particular optimisation technique. I think the real problem here is that it's still too hard to load a graph of objects out of a find_by_sql call. If that were fixed you'd be able to optimise cases like this, without too much complication to AR's codebase.

5

Michael Koziarski wrote:

I'm personally not sold on this particular optimisation technique. I think the real problem here is that it's still too hard to load a graph of objects out of a find_by_sql call. If that were fixed you'd be able to optimise cases like this, without too much complication to AR's codebase.

Michael, would you be able to expand on this?

6

He means there’s no way to do custom eager loading. Namely, a find_by_sql (or any find call without :include for that matter) only ever instantiates models of a single type. There is a much larger class of performance problems that can be solved by allowing instantiation of arbitrary models out of any given query (ie. model graphs based on dev-supplied JOINs).

Personally I believe this should be possible not only with find_by_sql, but also find + :joins/:select, since that allows the developer to continue leveraging the power of AR query generation which I rely upon heavily in my current project.

I’m working on a solution to this, but I have quite a few deadlines to get through before I can devote serious energy to it. The hard part is figuring out what the interface should look like.

7

Gabe da Silveira wrote:

He means there's no way to do custom eager loading. Namely, a find_by_sql (or any find call without :include for that matter) only ever instantiates models of a single type. There is a much larger class of performance problems that can be solved by allowing instantiation of arbitrary models out of any given query (ie. model graphs based on dev-supplied JOINs).

Personally I believe this should be possible not only with find_by_sql, but also find + :joins/:select, since that allows the developer to continue leveraging the power of AR query generation which I rely upon heavily in my current project.

I'm working on a solution to this, but I have quite a few deadlines to get through before I can devote serious energy to it. The hard part is figuring out what the interface should look like.

Gabe, are you talking about eager-loading finds having custom joins that don't just load along a chain but match-up arbitrary foreign keys? I and others have written patches for find_by_sql/:finder_sql that allows them to accept an :include option, instantiating the object hierarchy from SQL that uses field aliasing of the Rails eager loading form:

http://www.kellogg-assoc.com/articles/2006/11/05/eager-finder-sql http://mrj.bpa.nu/eager_custom_sql_rails_1.2.rb

Or are you talking about having eager-loading instantiate the cross and back links of the object hierarchy? A patch that implemented the first step of this -- matching and caching not just the instances seen in each eager-loaded association, but those in each model class -- was posted here several weeks ago by Fred Cheung.

A simple eager-loaded object tree is the most common query required in a Web app, and being able to restrict the attributes selected can make a huge speed and space difference when unwanted fields are large and the eager load runs deep.

8

Gabe, are you talking about eager-loading finds having custom joins that don't just load along a chain but match-up arbitrary foreign keys? I and others have written patches for find_by_sql/:finder_sql that allows them to accept an :include option, instantiating the object hierarchy from SQL that uses field aliasing of the Rails eager loading form:

http://www.kellogg-assoc.com/articles/2006/11/05/eager-finder-sql http://mrj.bpa.nu/eager_custom_sql_rails_1.2.rb

I think that the feature you've implemented there should probably be rolled in to active record at some point, whether or not that's for 2.0 I'm not sure.

However this plugin clearly minimises the need for the custom :symbol approach, or is there another case I haven't thought of?

> Or are you talking about having eager-loading instantiate the

cross and back links of the object hierarchy? A patch that implemented the first step of this -- matching and caching not just the instances seen in each eager-loaded association, but those in each model class -- was posted here several weeks ago by Fred Cheung.

A simple eager-loaded object tree is the most common query required in a Web app, and being able to restrict the attributes selected can make a huge speed and space difference when unwanted fields are large and the eager load runs deep.

Full bi-directional associations are probably too much to push out for 2.0, but in the long run it'd be nice to get them in there.

If finder_sql + :include can cover all the cases here, I think we'd be better off looking to merge that functionality into rails, rather than building some more syntax for a specific place we need it.