Approaches to improve response times for web pages who's HTML & content is loaded through Rails???

Hi,

I'm serving files (text, images) to a browser via my Rails application
as the user needs to be authenticated too. The files are on file
system, however references are in the database, as I'm using
PAPERCLIP. Using AUTHLOGIC for authentication. I'm finding the
response times aren't good, as each request for HTML/image etc have to
go through the Rails layers I guess of AUTHLOGIC, PAPERCLIP then
stream back.

QUESTION: Given I want users to authenticate for HTML & associated
files (CSS, images) then page needs, is there any obvious way to get
an order of magnitude improvement? E.G.

a) I guess I could save in session state the list of children files a
HTML page needs & get its file location path, until the subsequent
requests come, and that would some a database lookup (at the expense
of using session state)?

b) is there a way to keep open a TCP session or something for the
entire web-page load? (clutching at straws here)

c) any way to stream all html & images/css as one file/stream to
browser? (I assume not as this isn't how the browser works)

d) Combine Javascript into one file (I've heard of this one and can
follow up)

e) Set future expiry headers on files (I've heard of this one and can
follow up)

Thanks

PS. On average each request is about 3000ms to 5000ms when being
served by RoR (authlogic/paperclip), as opposed to about 160ms for the
same content but put directly under /public. I'm using mongrel.

I believe that paperclip actually stores a copy of the images in the
public directory by default. Once the html is rendered to the browser
the image requests should not go through the rails stack.

Let us know what firebug says. Browser load times are dependent on
many factors, and often the rails stack is not the slowest piece of
the pie. Once you prove there is some evidence that rails is
significantly slowing you down, then it would be worth trying to
optimize it. But minimizing javascript, css, and images often has a
more profound effect on browser load times. Then caching pages,
actions, etc helps immensely too.

Let us know the firebug or safari browser load time for each part of
the http requests.

Andrew

thanks - so I do want to have the images etc authenticated as well by
the way, so this is why I haven't got them under public either...

so the two test scenarios are based on doing a "save as" from a
website, and then:
(a) saving the web content directly under /public - which runs fast
(b) having the content in a protected area, and then having rails
serve it up via a controller - again that does (a) authentication via
authlogic and then (b) serves via paperclip which stores files on
disk, however there is a look to the database to find out where the
file is

I'm not sure how to attach the images here, however the results
basically show that on average, picking some of the images/css etc
that the browser fetches:
* a ~10kb javascript file: 10ms => 497ms
* a 1kb image: 5ms => 29ms
.
.
This is measured from a browser running on the same server as the ./
script/server -e production, mongrel server...

http://www.engineyard.com/blog/2009/your-pages-will-load-faster-with-rails/

Some good points in the link - but I'm thinking it still doesn't
address the large fundamental increase in response time when having to
go through rails for the authentication etc. In both cases in the
comparison for example there was no expires time populated.

I'm thinking maybe what i need to address is whether it's normal that
adding some extra code paths and a few database calls (authlogic and
paperclip lookup) should be taking this extra 300-400ms or so, or
whether there is something wrong here? Perhaps I'll need to under the
paperclip and authlogic code so I put some logging code in to measure
the times?

I am still not convinced the problem is with rails and not just
because there are a lot of images and files to load.

I like your thought to try to see exactly how long each process
takes. The terminal in development mode can be helpful too. It will
list the time for many of these things.

Here is another resource to help. http://railslab.newrelic.com/scaling-rails

I don't know the nature of the site, but if at all possible you should
try to use the http server to serve css, javascript, and images. You
could even resort to other forms of authentication, using apache if
you had to.

You can get a lot more specifics about what's happening time-wise
by installing the NewRelic monitoring plugin (http://newrelic.com/).

Highly recommended.

Are you re-checking authentication for each image? Could you find some
way to authenticate only once for all of your resources? Perhaps use
Basic or Digest authentication for your images, and then somehow tie
the password to the current session? If you can remove unnecessary
database hits, you may be able to shave off some ms. You may add a few
ms for HTTP authentication, though, so I'd take a close look.

Are you using a mongrel cluster? If this is all going through one
mongrel instance, it's only handling one request at a time. Switching
to passenger may help, as it can spawn new instances to handle
simultaneous requests (depending on settings and your server's
memory).

Thanks

* I'll try new relic then to get some more incite
* re authentication I'm using authlogic so in fact after initial
authentication it should use session cookies. However authlogic does
store sessions in database..
* re number of images etc and mongrel cluster, I'm only testing with
one mongrel, however I am here looking at per request stats
* re authenticating via apache: if I knew how I could do this, bit
have it integrated with my apps user provioning I'd look at this. Any
leads here? I couldn't find anything when I looked.

Thanks

tried new relic however it requires silver to do a web transaction
view, which seems to be a minimum $100 monthly even if I only want to
use it for 10 minutes...I bit expensive for a hobbyist programmer who
just wants to get time response time log points in to delineate times
spent in: rails generic code, authlogic, paperclip, database... I
might try to work out how to put logging into these points myself
first.

Any advice welcome on how/where to put logging code that would:
a) be the first bit of rails code hit when the request comes in?
b) first point called out to authlogic for authentication (perhaps
this is just in the specific controller before the
"before_filter :require_user" line, and one at end point?
c) first point for paperclip call (would be just in the controller
before teh "webfile = Webfile.find_by_path( search_path )" line
perhaps)?
d) point where all the above calls complete and the streaming of data
back to the browser starts (perhaps just in the controller at the "
send_file webfile.file.path , :type =>
webfile.file_content_type, :disposition => 'inline'" line)?

Mmmm. Haven't been paying attention, apparently their business
model has, er, evolved :slight_smile:

Alternatively, you might want to look at the rails-footnotes gem.

HTH,

PS I should say what the free screen did show from a high level point
for the call I made was:

* average response time broken down by tier
* ~40ms Ruby
* ~140ms Mongrel wait
* ~3ms Database