Sending files from RoR application

rubyonrails-talk
1

I need to send about 10Mbyte files to users. Am using Apache + mongrel_cluster. The straight forward way would be using acts_as_attachment with file system storage and just send it away.

Would there be any big performance problem with doing it straight forward?

Should I do something more adventurous via an Apache mod?

Thanks, Helzer

2

You could forward them to a different URL, esp. one that's not proxied through to the mongrel. If you need authentication you should think about using a combined php/ ror setup, where you have just one php-script, that checks for some cookie-based authentication and then delivers the file by readfile(). I may be wrong here, but afaik you will be blocking one mongrel- process for the duration of the download if you don't hand it off to a different delivery-method. so it depends on how many mongrel-processes you can spare for analog- modem-users :slight_smile:

hope this helps, phil

3

If properly set up, static files should be served directly through Apache and not touched by Mongrel at all, which means that a file being downloaded will not cause the mongrel process to block. In fact, you should be able to shut down your mongrel server and still have the files available for download (Apache will serve them directly)

The only complication is if you require file downloads to be authenticated, then they must either be sent through mongrel/rails using send_file, or you can use the X-Sendfile header with Lighttpd to send the file (X-Sendfile - lighty's life)

Adam

4

Hi Phil,

I may be wrong here, but afaik you will be blocking one mongrel- process for the duration of the download if you don't hand it off to a different delivery-method.

That can't be true. Mongrel received uploaded files when Apache finishes collecting them (all parts of a multi-part upload) and delivers files to Apache as a whole, for Apache to slowly send to the user. That's the way all content gets delivered (not just sent files).

I'm just thinking about the resources required by Mongrel to read the large files from disk, buffer them (fully) in memory and deliver to Apache, whereas Apache will probably read them chunk by chunk from the drive as users download.

Amir

5

Hi Adam,

I'm using Apache and want to send the files only to registered users (meaning, it needs authentication, and can't just sit in public/). Do you think that the resources required by Mongrel to read these files and send to Apache might become a big deal?

Thanks, Amir

6

Ah .. I see your problem but: - do you really expect to have 50 users downloading these files simultaneously ? (rough calculation: 50 * (10 M (file) + 12M (Apache thread-overhead)) ~ 1.1 GB Ram ( I'd say you need at least 2G anyway to be serving 50 people at the same time - even without file downloads) - do you really think using that much memory would slow down the server less than having to read from 50 files simultaneously ? - if you still want to ensure native delivery by apache use one of the many mod_rewrite-setups that will serve the content statically form the same virthost - will they be the same file or a different one for every user ? (because of apche/linux-filecaching) - will there be authentication needed ? if yes, i doubt you will get by without some mongrel/rails-call

I know I'm not helping much, but you really should benchmark your different solutions by using jmeter or similar.

Phil

7

If you need authentication then you'll be using rails and send_file which will tie up a process until the download completes. Like I mentioned before, you can use the X-Sendfile header as an alternative, which will send the file using your web server rather than rails. There's an article and a rails plugin available from this site: http://john.guen.in/past/2007/4/17/send_files_faster_with_xsendfile/

I've never actually used the X-Sendfile header method, since it's been a while since I've needed to serve protected data, so I'm by no means an expert on this, just passing along some information that I've come across which you might find useful.

Adam

8

Thanks Adam,

This looks like exactly what I need. I’ll run it tomorrow and let you know how it works for me.

Amir

9

A little update...

x_send_file is just fine. The install instructions are a little broken, but eventually it works. Get it from: http://tn123.ath.cx/mod_xsendfile/

After create the extension with: apxs2 -cia mod_xsendfile.c

the extension gets copied to some default apache modules directory, and not necessarily to where you need it. In my case it went to: /usr/lib/apache2/modules/mod_xsendfile.so

So, I copied that to my Apache modules directory, loaded it, and it runs just fine. Next, I need to change the Ruby plugin so that it only loads in production. Otherwise, my development and test environments don't download anymore (as I don't have xsendfile via mongrel directly). That should be simple enough to do with the RAILS_ENV variable.

The bottom line is, download contents gets served just fine with zero Mongrel effort.

Amir

10

helzer wrote:

A little update...

x_send_file is just fine. The install instructions are a little broken, but eventually it works. Get it from: http://tn123.ath.cx/mod_xsendfile/

After create the extension with: apxs2 -cia mod_xsendfile.c

the extension gets copied to some default apache modules directory, and not necessarily to where you need it. In my case it went to: /usr/lib/apache2/modules/mod_xsendfile.so

So, I copied that to my Apache modules directory, loaded it, and it runs just fine. Next, I need to change the Ruby plugin so that it only loads in production. Otherwise, my development and test environments don't download anymore (as I don't have xsendfile via mongrel directly). That should be simple enough to do with the RAILS_ENV variable.

The bottom line is, download contents gets served just fine with zero Mongrel effort.

Amir

Amir but how configure apche httpd.conf file so as apache handle this request not mongel in my case rails is handling request.. http://stackoverflow.com/questions/3778360/how-to-i-force-apache-to-handle-download-request-running-apachemongrel

is this what i post to stackoverflow my problem is mongrel handling this request not apche??