Dealing with large chunks of data

I'd suspect that the "data" you're iterating through is in the form of ActiveRecord models, yes? If so, you might get a big win if you can avoid that overhead completely by dropping into SQL and using something like select_all [1] (to get hashes back rather than model objects).

Perhaps a bit of code would let us help you better.


[1] Parked at Loopia

Rob Biedenharn