Uploading Large Files

I built an app that allows users to purchase video files to download, but unfortunately media temple's grid server isn't letting me upload large files. Everything goes great unless the files are over 100 mb. I'm using attachement_fu.

Is there a service or technique that can allow me to put large files on my server.

Sounds like a case for Amazon S3.

FTP them, then use the Ruby File classes to read them into your attachment_fu models. But my worry is how you’re going to serve those files if mediatemple won’t install the xsendfile module. If you’re going to stream those using send_file or send_data, you’re going to get into some serious problems, no matter how they serve their rails applications. I’d advise you to look into Amazon S3 since you seem to be charging for your videos anyway?

Best regards

Peter De Berdt

FTP them, then use the Ruby File classes to read them into your attachment_fu models. But my worry is how you're going to serve those files if mediatemple won't install the xsendfile module. If you're going to stream those using send_file or send_data, you're going to get into some serious problems, no matter how they serve their rails applications. I'd advise you to look into Amazon S3 since you seem to be charging for your videos anyway?

Best regards

Peter De Berdt

Hey Peter - this was posted a long time ago...hopefully you are still following this thread, but I was wondering what you mean by "you're going to get into some serious problems"? I'm actually using send_file and my Ruby on Rails Container keeps terminating because it has "run out of Ram". This happens when the system is using the send_file function for large files. This may have been what you previously warned me about. Do you have any advise or techniques to overcome this issue?

I ended up doing the FTP thing as you advised, but failed to take into consideration your send_file warning. Let me know if you have any ideas.

thanks...

send_file and send_data will load the entire data stream into memory before serving it, that’s why your app goes down due to excessive memory use.

Now, as I said, there are a couple of options you might consider:

  1. Use Apache’s or nginx’ XSendFile module to serve the files. If you’re using some kind of shared hosting, where you have no control over the actual web, then this method is gonna fall flat on its face, because they probably won’t want to install that module for you. Apache will then serve a file with a temporary url.

  2. Use an external hosting service like Amazon’s S3 to serve the files, where you basically offload the file serving to a service that’s made just for doing so. Attachment_fu should be able to help you out there.

Best regards

Peter De Berdt

thanks paul...

one last question - I've got over 18 gigs of video files on my media temple server that I will need to store on the amazon s3 storage. What's the best way to transfer these over?

If you have shell access to you media temple account and can run ruby on it, using s3sync could be an option:

http://s3sync.net/wiki

Haven’t used it, but seems to fit your needs.

Best regards

Peter De Berdt