Direct Upload with S3 & Multipart

Hello all,

Recently I’ve been developing a file upload, and the use case I was presented required the files not to pass through the application, but to use DirectUpload from ActiveStorage so it goes directly to an S3 bucket.

The max file size I was suggested was 8GB, S3 allows up to 5TB but only in multipart mode, in single part (what’s in place right now in active storage) is only 5GB max.

We managed to work out the limitation, but we’d be ideal to be able to get to the maximum S3 can provide.

I’ve been reading a bit the code, and the modifications aren’t that wild, but have no knowledge if other services like Azure or Digital Ocean have the same support for big files using multipart mode.

What do you think?

Thanks for reading!

I’ve documented a bit the current state and a proposal with sequence diagrams (not 100% accurate, but good enough to grasp the idea)

Current State:

Proposal (to support multipart uploads):