massive data output


i can't speak for rails specifically, but in general there are two places this is going to choke.

1. the server. if the server tries to load the whole query before rendering it and passing it off to the client, it will have to build up a huge buffer. in environments where the user has more control over rendering, the usual solution is to render the header and then start reading from the database and writing out at the same time, then tacking the footer on the end. i do not believe this is possible under rails, although there may be a way i'm not aware of.

2. the client. browsers really don't like rendering big tables, since they have to have the whole table in memory before they can draw it. the solution to this might be to get rid of the big table: move to css based layout, or replace the big table with a lot of smaller tables (e.g. one single-row table per item). that's still going to leave you with a several megabyte file, though, so the browser may hate you anyway.