Upload large video to Digital Ocean

rubyonrails-talk
1

I use Active store using the S3 API to connect with Digital Ocean Spaces to handle video files. Direct upload is set to false

For files smaller than 300 M, it works perfectly.

For files between 300 MBytes and 400 MBytes, it works sometimes and it does not some other times.

I use Rails 7.0.3.1 with import map.

For every file larger than 500 MBytes, the upload stops without any error message and the form is reloaded.

Anyone has a suggestion?

2

Hey Yanik,

What are you seeing in your log files since it’s just reloading the form? Did you override the multipart_threshold at all? S3: permit uploading files larger than 5 GB by georgeclaghorn · Pull Request #35931 · rails/rails · GitHub

Are you seeing any OOM calls? Usually just running dmesg or looking at the web server/service start times you can see if that’s the case.

3

This is the problem. There is nothing in the log file. The upload starts (a 1Gbytes video), the Create/Update method (in the Controller) is not called, I see the progress bar running and then the page is reloaded .

For smaller files ( 230 MBytes or less), the upload starts and when the progress bar reaches 75%, the Update/Create method is trigged, including a delayed ActiveJob used to analyse the video (metadate[:duration] is not immediately available) and delete older video not used anymore (replaced by the one just uploaded) and then the progress bar continues to grow until it reaches the right hand side of the window (with the browser inspector, the progress bar could go up to 130%).

4

And this is with debug logging turned on?

5

With config.log_level = :debug set:

Started GET "/documents/23/edit" for 69.165.196.51 at 2022-08-05 11:06:56 -0400
...
Completed 200 OK in 30ms (Views: 25.5ms | ActiveRecord: 0.7ms | Allocations: 34667)

(filling the form ... 40 seconds)

Press the save button
-- wait 30 or 40 seconds **nothing happens in the console**

Started GET "/documents/23/edit" for 69.165.196.51 at 2022-08-05 11:08:35 -0400
6

… but I have found a way to keep the browser console and I have found this:

POST https://.../documents/23/ 413 (Request Entity Too Large)

7

There it is! So just something on the web server config side. Nginx? Apache?

8

I just figured out. Thank you for your help.

1 Like