AR Result Set plugin

Hi all,

A short while ago some of us were wondering on rails-contrib/here about the feasibility of Base.find returning a proxy object and cool things that we could do with that. I've been toying with a small plugin that adds some of that stuff. It's now in a state where it might be fun for people to play with (Rails' unit tests basically past with this plugin loaded. There are a few failures, but nothing super interesting)

What there's right now:

posts = Post.find :all
posts.load :comments => :author #preloads the comments association and the author from comments. Same syntax as your normal :include/ :joins conditions

posts.comments.first #=> triggers an eager load of comments on all posts

posts.comments.first.author #=> triggers an eager load of the author of all comments

Of the quirks I am aware of/anticipate the biggest cause is the fact that in some cases eager loading a collection behaves slightly differently to loading it normally. (for example :order on has many through is a bit off). With this plugin installed basically every association load is an eager load so no hiding from any of those issues.

There are interesting questions around garbage collection: because each record references the collection it was part of, keeping one of them around keeps them all around. Playing with WeakRef is a possibility (but right now that would actually break some of the functionality). Detaching all objects with each request cycle is another as is figuring what patterns indicate that the user probably does want to detach the record and do that automatically. I feel that picking one of the options right now would be premature.

Caveats/Gotchas/etc:
- Requires Rails 2.1
- Anything before http://github.com/rails/rails/commit/b7a37b742c0abd1df8ea48cc82f76385cc0c41ea (sometime last week) will die horribly if you try and load a has_one through (because preload of has_one :through is borked before that).
- has_many associations with a :limit option are excluded because they can't be eager loaded nicely.
- has_many :through with options/conditions on the join table will probably be screwed up:

has_many :foos, :conditions => ...
has_many :bars, :through => :foos, :conditions => something only on bars

should be ok

has_many :foos
has_many :bars, :through => :foos, :conditions => something on foos

probably won't work

Code:
http://github.com/fcheung/ar_result_set/tree

I'm at railsconf so grab me/ smack me on the face if you feel so inclined

Fred

Just to follow up on this (not that I've had time to touch this since last week), what do people think we could do with this that would be useful ?

Fred

It might be useful for lazy loading from the database inside cached
view fragments. One would normally only want to hit the database if
the data inside a view fragment isn't in the cache.

Have you had any feedback on RailsConf?

Ah wait, I just realized that such a proxy could be useful in very
large result sets. The proxy could only fetch a small amount into
memory at a time, e.g. by using cursors.

Just to follow up on this (not that I've had time to touch this
since
last week), what do people think we could do with this that would be
useful ?

Fred

It might be useful for lazy loading from the database inside cached
view fragments. One would normally only want to hit the database if
the data inside a view fragment isn't in the cache.

Have you had any feedback on RailsConf?

Yeah, I chatted to Koz about it. I'm planning to play around with
identity maps and stuff like that in conjunction with that (not that I
have very definite plans - more of a lets toss all these things in the
air and see what happens :slight_smile: )

Ah wait, I just realized that such a proxy could be useful in very
large result sets. The proxy could only fetch a small amount into
memory at a time, e.g. by using cursors.

That's certainly a thought (but I have to confess ignorance as far as
cursors go, beyond the dictionary definition)

Fred

http://www.postgresql.org/docs/8.1/static/plpgsql-cursors.html

On the other hand, I don't know what identity maps are. :slight_smile:

Yeah, I chatted to Koz about it. I'm planning to play around with
identity maps and stuff like that in conjunction with that (not that I
have very definite plans - more of a lets toss all these things in the
air and see what happens :slight_smile: )

Ah wait, I just realized that such a proxy could be useful in very
large result sets. The proxy could only fetch a small amount into
memory at a time, e.g. by using cursors.

That's certainly a thought (but I have to confess ignorance as far as
cursors go, beyond the dictionary definition)

http://www.postgresql.org/docs/8.1/static/plpgsql-cursors.html

thanks

On the other hand, I don't know what identity maps are. :slight_smile:

Quite simple in theory - don't have more than one instance of an activerecord object per row in the database. Plenty of edge cases in practise :slight_smile:

Fred

Hongli Lai wrote:

I was wanting a result set proxy just recently to hang stuff like sorters off off. Might be useful in conjunction with named_scopes.