Active Storage in production: lessons learned and in-depth look at how it works

[brenogazzola]

1. Introduction

In my company, we lucked out that Rails 5.2 (and Active Storage) was released just before we needed to implement user uploads. This means that we’ve been using it in production for over 5 years, across 3 different hosts (Heroku, AWS, and GCP) and 3 different storage providers (S3, GCS, and R2).

Our primary use case is image galleries, either of products we sell, or user-uploaded images for their wedding/birthday/baby shower websites. This means that we rely heavily on image transformations, which Active Storage makes trivial to generate.

While we love Active Storage (which I’ll call AST from now on), there are certain design decisions in AST that one should be aware of, as they have an impact on the overall performance of the app, and not just the pages that use AST.

2. Understanding Active Storage

2.1 The basic use case we will use as an example

To understand some of the things I’ll talk about below, we must first take a look at how active storage works under the hood. Let’s assume you have a Company model and want to use AST to store its logo:

class Company < ApplicationRecord
  has_one_attached :logo
end

You can attach a file by doing this:

@company.logo.attach(io: File.open(Rails.root.join("public/favicon-192.png")), filename: 'logo.png')

And then display an optimized version of it, which AST calls “variant”, by doing this:

<%= image_tag @company.logo.variant(resize_to_limit: [200, 200], saver: { strip: true, compression: 9 }) %> 

Open the page and you will see your company logo has been resized to be 200px in width and 200px in height, and compressed to reduce data usage. No need for extra columns in your company table. Just say you want a variant and you have it.

2.2 Tables and columns created by Active Storage

When you install active storage, it’s going to create three new tables in your database.

2.2.1 active_storage_blobs

This table contains information about the file you have just attached:

• key: Name your file will have in the storage serice. It’s generated by the generate_unique_secure_token method in ActiveStorage::Blob. It contains only numbers and lowercase letters;
• filename: Name your file originally had when you attached it;
• content_type: Extracted by the marcel gem and based on binary data, declared type and filename;
• metadata: Extracted by the analyze method in ActiveStorage::Blob. It can tell various things about the file, like height, width, rotation, etc.;
• service_name: The storage service your file was uploaded do. This was not part of the original implementation, but was added later to support having a different service per attachment;
• byte_size: Size of the file in bytes;
• checksum: You will never have to worry about this one. AST uses it to ensure file is not corrupted.

2.2.2 active_storage_attachments

This is the join table used to connect the blobs (files) to your models.

• blob_id: Reference to active_storage_blobs;
• record_type and record_id: Polymorphic reference to your model. In our example record_type = 'Company';
• name: The has_one_attached attribute. In our example name = 'logo'.

2.2.3 active_storage_variant_records

This table tracks which variants have already been generated. It was not part of the original implementation, but was added later as an optimization to avoid having to check if the variant file existed in the storage service.

• blob_id: Reference to active_storage_blobs. This is NOT the variant’s blob, but the original blob;
• variation_digest: Generated from the options we gave the .variant method that created this record. In our example, this would be the digest of the hash { resize_to_limit: [200, 200], saver: { strip: true, compression: 9 } }.

2.3 How it all works under the hood

2.3.1 When you attach an image

1. AST creates a new record in active_storage_blobs (B1);
2. AST creates a new record in active_storage_attachments (A1) connecting the newly created blob (B1) to the company;
3. AST uploads the file to the storage service, using the name generated by the blob (B1);
4. AST enqueues an ActiveStorage::AnalyzeJob to process the blob (B1) and extract metadata from it.

2.3.2 When you display an image

When you display the original file (not a variant) in an image_tag:

1. AST looks for an active_storage_attachment (A1) for your company, with name = "logo";
2. AST looks for the active_storage_blob (B1) the attachment references;
3. AST generates a URL that points to one of its own controllers, not the storage: /rails/active_storage/blobs/:signed_id/*filename.

When you display a variant in an image_tag:

1. AST looks for an active_storage_attachment (A1) for your company, with name = "file";
2. AST looks for the active_storage_blob (B1) the attachment references;
3. AST generates a URL that points to one of its own controllers, not the storage: /rails/active_storage/blobs/representations/:signed_id/:variation_key/*filename.

When the browser requests the URL of the original image:

1. The request is routed to ActiveStorage::BlobsController#show (pre Rails 6.1, there are now two possible controllers, see in 2.3.3);
2. AST looks for the active_storage_blob (B1) in the signed id;
3. AST redirects the request to the URL of the generated file in the storage service.

When the browser requests the URL of the variant image for the first time:

1. The request is routed to ActiveStorage::RepresentationsController#show (pre Rails 6.1, there are now two possible controllers, see in 2.3.3);
2. AST looks for the active_storage_blob (B1) in the signed id;
3. AST decodes the variation_key into a hash of options (in our example it would be { resize_to_limit: [200, 200], saver: { strip: true, compression: 9 } });
4. AST checks in active_storage_variant_records and finds nothing, which means it has not processed this variant yet;
5. AST downloads the file from storage;
6. AST creates a new record in active_storage_variant_records (V1, which references the original blob);
7. AST creates a new record in active_storage_blobs (B2) for the variant it’s about to generate;
8. AST creates a new record in active_storage_attachments (A2) which references the new blob (B2) and the variant record (V1);
9. AST passes to the image_processing gem the options from step to 2 and the downloaded file, causing it to generate a new file;
10. AST uploads the newly created file to the storage using the key specified in the blob (B2);
11. AST redirects the request to the URL of the generated file in the storage service.

When the browser requests the URL of the variant image after the first time:

1. The request is routed to ActiveStorage::RepresentationsController#show;
2. AST looks for the active_storage_blob (B1) in the signed id;
3. AST decodes the variation key into a hash of options (in our example it would { resize_to_limit: [200, 200], saver: { strip: true, compression: 9 } });
4. AST checks in active_storage_variant_records (V1) and finds it;
5. AST looks for an active_storage_attachment (A2) that references the variant record (V1);
6. AST looks for the active_storage_blob (B2) the attachment references;
7. AST returns the URL of the generated file in the storage service.

2.3.3 Serving files through rediret mode, proxy mode and public mode

Until Rails 6.1 Active Storage could only serve images using redirect mode. That is, it generated a URL to one of its own controllers (blob for original files, representations for variants), and that controller redirected the request to a signed, expirable URL to the actual file in storage. While this worked fine for private files, it meant if you were serving public images your CDN could not cache them.

Thanfully, two new modes were added by contributors:

1. Proxy mode: Instead of redirecting the request to the storage, AST will stream the file requested. This allows your CDN to cache your images, but keeps one of your puma workers busy while the file is being streamed.
2. Public mode: This changes the url generated so that instead of pointing the an AST controller, it points directly to the file in storage. This means no extra load on your app when image is requested, but extra load (in the form of queries) to generate the URL of variants when the view is being generated.

Here’s a table explaning each controller used and URL generated:

mode	image	controller	url
redirect	original	ActiveStorage::Blobs::RedirectController	/rails/active_storage/blobs/redirect/:signed_id/*filename
redirect	variant	ActiveStorage::Representations::RedirectController	/rails/active_storage/representations/redirect/:signed_blob_id/:variantion_key/*filename
proxy	original	ActiveStorage::Blobs::ProxyController	/rails/active_storage/blobs/proxy/:signed_id/*filename
proxy	variant	ActiveStorage::Representations::ProxyController	/rails/active_storage/representations/proxy/:signed_blob_id/:variantion_key/*filename
public	original	-	varies depending on the storage service (S3, Google Storage, etc.)
public	variant	-	varies depending on the storage service (S3, Google Storage, etc.)

2.3.4 The direct upload flow

When allowing your user to upload photos, there are two ways to use to built-in file method in the default builder:

<%= form.file_field :attachments %>
<%= form.file_field :attachments, direct_upload: true %>

The first one is easier to use. It will upload the file to your app, and then your app will upload it to the storage service. The second one is more complex because it requires you to configure CORS in your storage and if you are building an SPA you will probably need to write custom javascript code to handle the upload, but it has the advantage of not requiring your app to handle the file upload (I will discuss why that is important in 2.4).

When using the direct upload method, the flow is as follows:

1. On form submission Turbo notices the direct_upload instruction and halts the submission;
2. Turbo sends a POST request to ActiveStorage::DirectUploadsController#create;
3. AST creates a new record in active_storage_blobs (B1) for the file;
4. AST returns a JSON containing information about the blob and a signed, secure URL that Turbo should use to upload the file;
5. Turbo uploads the file to storage and inserts the signed_id it got from AST in the value of the file field;
6. Turbo resumes the form submission;
7. When the apps controller receives the params and uses them to save/create the model, AST grabs the value it got in the file field and uses that to find the blob (B1);
8. AST creates a new record in active_storage_attachments (A1) which references the blob (B1) and the model;
9. AST enqueues the ActiveStorage::AnalyzeJob to process the blob (B1) and extract metadata from it.

2.3.5 The PNG fallback

Among the many config options in AST, there are two that are important if you are displaying images:

• web_image_content_types: The image formats you are willing to serve to your users. By default it contains only png, jpeg and gif.
• variable_content_types: The image formats the version of libvips/image_magick installed in your servers can handle.

Together these two control AST behaviour when you ask it to display the original version of the image. If the image uses a format listed in web_image_content_types, then AST will serve it as-is. Otherwise, it will check if the format is listed in variable_content_types and if yes, it will convert it to PNG, since it believes that your image library can handle the conversion. Finally, if AST does not find the format in either list, it will serve the image as a binary file.

In practice, there are two changes you need might need to make to these lists:

• If you want to serve webp images, you need to add it to web_image_content_types;
• If you are using a version of image_magick/libvips from your distro repos, there’s a pretty good chance it does not support converting some of the formats listed in variable_content_types, so you should remove them, otherwise your exception handling service (Sentry, Honeybadger, etc.) is going to get flooded with errors.

2.3.6 The analyze job

Every time a file is attached to a model, AST will enqueue an instance of ActiveStorage::AnalyzeJob, which in turn will call the analyze method on the blob. This method is used to extract metadata from the file, and it does that by passing the blob to one of the various analyzers available by default in AST:

• ActiveStorage::Analyzer::ImageAnalyzer: Extracts width and height;
• ActiveStorage::Analyzer::VideoAnalyzer: Extracts width, height, duration, angle, aspect ratio and if the file contains audio and video channels;
• ActiveStorage::Analyzer::AudioAnalyzer: Extracts duration, bit rate and sample rate;

While these are usually good enough, you can easily write your own analyzer to extract more information from the file. For example, let’s say you want to know if a file is transparent to figure out if it’s safe to convert a PNG into JPG to reduce its size, and you are using libvips:

class MyAnalyzer < Analyzer::ImageAnalyzer::Vips
  def metadata
    read_image do |image|
      if rotated_image?(image)
        { width: image.height, height: image.width, opaque: opaque?(image) }.compact
      else
        { width: image.width, height: image.height, opaque: opaque?(image) }.compact
      end
    end
  end
  
  private
    def opaque?(image)
      return true unless image.has_alpha?
      image[image.bands - 1].min == 255
    rescue ::Vips::Error
      false
    end
end

Then just add it to the analyzer config array:

Rails.application.config.to_prepare do
  Rails.application.config.active_storage.analyzers.prepend MyAnalyzer
end

2.4 What all of this means for your app

Now that we know how AST particular brand of magic works under the hood to allow attachment images and creating variants without needing new columns or tables, here’s a few rules of thumb and things to watch out for, no matter if you are handling images or every type of file.

2.4.1 You have to very careful about N+1 queries when displaying multiple records with attachments

As we saw above, to display the logo of your company, AST has to make to extra queries, first to find the attachments (the join table), and then the blob that is the actual file. To avoid that, you can either use the available scope or do it by hand:

@companies = Company.all.with_attached_logo
@companies = Company.all.includes(:logo_attachment, :logo_blob)

To make it clear, the pattern is with_attached_ATTRIBUTE and includes(:ATTRIBUTE_attachment, :ATTRIBUTE_blob), where ATTRIBUTE is the name you gave to has_one_attached.

2.4.2 Always use the direct upload version of file upload to protect your app from slow clients

When a user is uploading a file to your server, instead of directly to storage, it is keeping one off your puma workers busy, and preventing it from serving other requests. If the user has a slow connection, or is uploading a large file, this can take a long time.

If you are running with only a few web servers with low concurrency (say, you are in Heroku, running 2 standard 2x web dynos) you might only have 4 of those workers. If two users are “slow clients”, you just halved your capacity for as long as they are uploading their files. Which means that your other users might take longer to navigate between pages, causing frustration and a bad experience.

2.4.3 If you are using proxy mode, make sure you have nginx or cloudflare between your servers and your users to protect your app from slow clients

Same reasoning as above. Proxy mode in AST keeps a puma worker busy while the file is streamed from storage to the user. If the user is in a slow connection, that might keep your worker busy for a long time.

By having nginx or cloudflare in front of your app, you can configure them to buffer the stream allowing your worker to quickly send them the entire file and start serving other requests, while they handle the slowness of the client. Something they are much better equipped to do than Puma.

2.4.4 The on demand variant generation is a great feature, but it can bring your entire app to its knees if you are not careful.

Once again, let’s assume your app is small, or you prefer to scale horizontally, so your servers only have 1-2 vCPUs and not much memory (say 1GB). This means puma is probably configured for 2-3 workers and your app is using 80-90% of the memory.

In your app there’s a page where you allow users to upload multiple photos, and after the upload is done, they are redirected to their gallery, where you display smaller, compressed versions of their photos.

One of your users just uploaded 10 photos at once. After the redirect, their browser will request the smaller, compressed version of those 10 photos. Since these variants have never been generated, your servers will download and process them all at the same time. This means that you will have multiple variant generations competing for their limited CPU time, and eating whatever they had left of memory, causing them to start swapping.

This will cause a major slowdown not just for the image generationg requests, but for every other requests as they now struggle to get CPU time to execute their code, and have to use swap memory to allocate objects. I remember our APM showing pages with sub 100ms response times going past 200-300ms because even a simple .find was taking 50ms while image magick was processing a PNG.

Also, if you had less then 10 workers total, some of those image requests will be waiting in the load balancer’s queue for chance to be generated. And right behind them will be all the other navigation requests of your other users.

There are a few ways to mitigate this:

1. Replace image magick with libvips. It’s faster and uses less memory;
2. Scale your servers vertically instead of horizontally. It reduces the chance of all workers in a single servers being occupied and the extra vCPUs will process the image faster;
3. If you are close to the memory ceiling give your servers more memory to avoid swapping. In AWS this means switching families (c6i to m6i) and on GCP using a custom machine to add 1GB ram.
4. Generate your variants in advance, through a background job.

3. Running in production to serve images

Now that we understand how AST works, and the impact its design decisions have on your app, let’s talk about some lessons we learned from running it in production.

Warning: This part is a bit of a rant, and parts of it are about handling images in general, not AST specifically.

ImageMagick is too slow, uses too much memory and has too many security issues…

That’s why Rails 7 switched to libvips by default. If you want a more in-depth explanation, check my Make Vips the recommended/default variant processor for Active Storage thread here in the forum and compare the list of known CVEs for vips and known CVEs for ImageMagick.

… and the version of the image libraries in your distro’s repos is too old and doesn’t support enough file formats

This was one of the main drivers for us to leave Heroku. We were stuck using 18.04 LTS and had no way to try to get a more recent version of libvips installed. Fortunately, this is less of a problem now in 2023, where every LTS distro has a recent enough version of libvips in its repos.

What’s still a problem is file format support. The packages available in all distros support JPEG, PNG, BMP, TIFF, GIFs and, if you install the right package, WEBP. And that’s it. But you know users… gotta catch 'em all. Users can and will upload HEICs, AVIFs, JPEG2000, JPEGXL, and whatever else they can find.

So, how do you support those formats? You install the dev packages of every file format you need, then you compile your image library (ImageMagick or libvips) from source so that they can properly link to said packages.

Yes, compile from source.

And since you’re already doing that, I recommend you also remove libpng and libjpeg-turbo from your system and replace them with the faster libspng and the much better mozjpeg (this last one is important. I’ll talk about it later).

Your users will laugh at your poor attempts to restrict image formats using accept='image/jpeg,image/png' and upload whatever they want…

I haven’t figured out how they do it, but it seems that one of the Android browsers completely ignores the accept attribute (or at least the accepted image formats) and allows the user to upload whatever they want. It’s one of the reasons we’ve had to add support for so many file formats in our image libraries.

… and as a bonus, just because it’s a video file it does not mean it contains a sound stream… or a video stream

Yup. Android’s voice recorder uses a file format that is identified as video. So if you’re doing video manipulation in your web app, be aware you might get things you think are a video, but they don’t actually have a video channel, so make sure you check your blob’s metadata

Your CDN will not keep your images in their cache despite what they say…

You checked Cloudflare’s documentation and ran a few tests and noticed that the first two times you request an image you get a cache MISS, and the request hits your app. But on the third time, you get a HIT and the request does not hit your app. So you think you’re good, right?

Wrong.

All this means is that the data center (PoP) serving your requests has it in its cache. But CDNs don’t have a single data center. They have dozens, all around the world. Just in the US, Cloudflare has 46 of them. And unless you’re paying for a premium plan (or addon like Argo) where the cache is automatically propagated between PoPs, each one of them is going to make two requests for that image.

So just in the US, that’s 92 requests that will hit your Puma servers before the image is fully cached. Then all is well, right?

Wrong again.

You see, just because the image is in the cache, it doesn’t mean it will stay there. No, it does not matter that you set its TTL for 1 month since retention and freshness are different things. If your image is not requested often enough in a specific PoP, it will get evicted. Which means that PoP will let two more requests hit your servers.

Sure, your JS and CSS files, as well as your logo and maybe the images on your home page, have enough requests to keep them in the cache. But if you’re running an image-heavy web app? You’re going to have a bad time. If you’re running with only a few servers, you will definitely feel it in your request queue times (you are tracking those, right?) when a user decides they like half a dozen products on your list, opens each one in a new background tab, and suddenly a few dozen requests are hitting your proxy controllers (I really hope those variants are already processed).

… and nginx can solve the problem, but it has a pretty massive footgun if you are not careful

You can configure nginx to store any files served by your app (including those streaming by the proxy controller) on its disk, and serve them directly from cache when they are requested again. This keeps your puma workers free to handle other requests (and protected from slow clients).

However, when you are configuring it to do so, make absolutely sure that you are using proxy_hide_header and proxy_ignore_header on the Set-Cookie header. If you don’t do that, nginx will cache the session cookie, which your app might be using to store sensitive information, such as which user is logged in.

What does that mean? It means that user A is logged in, and navigating normally, and suddenly they are served a cached image with the set cookie of user B. And now user A is logged in as user B. I don’t have to tell you how bad that is, right? Especially if user B is an admin, right?

Serving properly optimized images is hard…

So here’s my recommendation: Resize them to 2x (that is, if you are going to display them as 100x100, resize them to 200x200), since anything higher is unecessary. Then apply one of the transformations below.

If you are using Vips:

# JPEG
user.avatar.variant({ saver: { strip: true, quality: 80, interlace: true, optimize_coding: true, trellis_quant: true, quant_table: 3 }, format: "jpg" })

# PNG
user.avatar.variant({ saver: { strip: true, compression: 9 }, format: "png" })

# WEBP
user.avatar.variant({ saver: { strip: true, quality: 75, lossless: false, alpha_q: 85, reduction_effort: 6, smart_subsample: true }, format: "webp" })

If you are using ImageMagick:

# JPEG
user.avatar.variant({ saver: { strip: true, quality: 80, interlace: "JPEG", sampling_factor: "4:2:0", colorspace: "sRGB", background: :white, flatten: true, alpha: :off }, format: "jpg" })

# PNG
user.avatar.variant({ saver: { strip: true, quality: 75 }, format: "png" })

# WEBP
user.avatar.variant({ saver: { strip: true, quality: 75, define: { webp: { lossless: false, alpha_quality: 85, thread_level: 1 } } }, format: "webp" })

Two things about those options above:

1. Make sure you add the format keyword. Recently some android phones have started sending images with a .jfif file extension, and those break libvips, even though they are normal JPEGs. By adding format: :jpg you are telling it its fine to treat it as a JPEG.
2. The options optimize_coding, trellis_quant and quant_table will only work if your image libraries have been compiled against mozjpeg. If you are using the default libjpeg, they will be ignored. They will also bring a further 20% to 35% reduction in my experience, without any noticeable loss in quality. This is enough to put JPEG on par with WEBP for many images.

… and Lighthouse/PageSpeed are going to try to bully you into a few bad choices

When it comes to images on your pages, there are two things that Lighthouse and PageSpeed will try to convince you into doing: using WEBP and resizing the images to the size at which they will be displayed.

Those are not always good advices. WEBP does not support interlacing, so the image is not displayed until it’s been fully downloaded. JPEG, on the other hand, does, so even if its file size is larger (and it might not be if you’re using mozjpeg), users will perceive it as loading faster because a low-resolution version will show up faster than the full WEBP version.

And when it comes to resizing images, it’s better not to go overboard. Let’s say your app has a product page where you display a large version of the photo along with a list of thumbnails that the user can click to view other photos. It’s better to let the thumbnails use the same variant as the large photo since that means that when the user clicks on them, the file will already be in their browser cache and be displayed immediately.

Every storage service has their own quirks that will drive you insane…

The S3 gem is configured to auto retry failures and has timeouts set to 60 seconds by default, which meant it spends minutes trying to complete a single upload, keeping a worker busy.

Google Cloud Storage does not like if two requests try to update metadata at the same time, which will break your code if you try to run a .analyze at the same time the ActiveStorage::AnalyzeJob for a blob.

R2 is still in its infancy, and in the couple of months since we’ve started using it, we’ve already had a few temporary connection problems.

… and if you want to migrate from one to another you and the company’s accountant are going to hate the person who made that decision

Unless you are using R2, you are going to be charged per GB transferred. And since AST does NOT allow you to split your files into folders, that means you have no way of differentiating between original files (which you might have no way of acquiring again) and variant files (which you can generate again). Therefore, you will have to pay to transfer all of them.

Bonus 1: If someone were to write ‘Active Storage, the good parts’, then has_many_attached would NOT be in it.

Every time we used has_many_attached we’ve regretted it. Not because it’s a bad feature, but because we always end up needing to do add some extra information to those files, which you can’t do when they are blobs.

For example, you product has many images, so you do this:

class Product < ApplicationRecord
  has_many_attached :images
end

Seems obvious. You can even do product.images.first to display a cover image. Except, a month from now you are taking a look at the page that lists every product you have available, and you notice that in one of the clothing articles the image is a size table. You want to choose another image as the cover image, but you can’t because they are blobs, and you can’t (shouldn’t) add extra attributes to them. So you have to redo your models:

class Product < ApplicationRecord
  has_many :photos
end

class Photo < ApplicationRecord
  has_one_attached :file
  validates :position, presence: true
end

And after you do that you realize you now have to migrate possible millions of images from simple blobs into full models. So do yourself a favor and stick to has_one_attached.

Bonus 2: You might need a larger connection pool than you think

Recently in the CGRP Slack group there was a discussion about an app throwing ActiveRecord::ConnectionTimeoutError even though its pool was configured to be equal to Puma’s thread config. @tekin.co.uk dug down until he found out it was being caused by Active Storage’s proxy mode. Check out the post on his blog for more details.

[vincentrolea]

This is one great summary of ActiveStorage inner workings as well as shortcomings. There’s a lot of stuff I can relate to in my own experience.

I’ve come to realize that AS is great when it comes to uploading (apart maybe from the JS part of direct uploads) but quickly becomes challenging when it comes to serving the uploaded files. In apps where a lot of images is served, I tend to turn to dedicated services in the likes of ImgProxy to remove some of the load on the server, since, as you said, we do not want to block a Puma workers to serve transformed images.

[brenogazzola]

I agree 100% with that. I’ve considered using a hosted ImgProxy and Thumbor, but not having an obvious way to integrate it with AST or adding support for more file formats and mozjpeg has been holding me back.

[hakunin (Max Chernyak)]

I enjoyed reading this even though I currently don’t use AST. In the past used to work heavily with images, and dealt with some of these issues, as well as some craziness with vector graphics. Like a 2kb purple rectangle svg that could bring down a 64GB RAM server.

A lot of this could be super helpful in the official Rails guides. I’m not on the team, so this is just a personal opinion.

[brenogazzola]

That’s crazy. Was that a random SVG or an intentional image bomb?

Also, glad you liked .

[hakunin (Max Chernyak)]

It could’ve been intentional. I had a dir on my desktop with wtf images uploaded to our servers, and used to test various image conversion tools against them. I seem to recall that particular image was bombing when processed with whatever ImageMagick was using for vector under the hood, but was handled just fine with a different tool we switched to for vector.

[ajaynomics]

This needs to go in as a new section in the rails guides.

[Spike]

The advice on has_many_attached should be shouted from the rooftops. Very seldom does it make sense to use it. I’ve only ever used it to link files uploaded into a WYSIWYG editor to the model that contains the HTML so that they can be deleted when the last page that uses the blob gets deleted or the last reference to the file is removed from the last page that was referencing it.

[Spike]

I’d also be interested to hear how others handle things like data recovery in the case of intentional (but mistaken) model removal where the model has attachments. Currently I have S3 configured to soft delete files and then eventually delete them after a half a year. It’s possible to then restore records from the database but the process of restoring the files is laborious.

I wondered if it’d be better to set AS to unlink the blobs rather than delete them when the related model is deleted and then change our AS garbage collector to delete unlinked blobs after half a year rather than a week.

[lazaronixon (_N_I_X_O_N_)]

The question that doesn’t have answer… Why Active storage doesn’t have validation for type and size? Security consultants always ask us to do that.

Is it the reason that, there’s no secure way to check a file size or type without upload the file first right?

[brenogazzola]

Best guess…

If you are doing direct uploads, size validation must be done is JS (can be bypassed) or the storage service (not something that can be handled by Rails).

Type validation good enough for security purposes cannot rely in headers or extensions, it needs to check binary data, which means downloading the file, which is in itself a risk for your server I think?

[santib (Santiago Bartesaghi)]

Hey @brenogazzola this is the best post about ASt I’ve ever read, so congrats for that BTW I think you should talk about it on RailsConf 2024

Also, WDYT about this part of the ASt guides? For me it’s crazy having to create those weird routes just so that the CDN host is used when generating routes with route helpers.

I’ve seen you contributing in the Rails codebase before, so I’m wondering, do you have ideas on how ASt design can be adapted so to help with the shortcomings and challenges that you described before?

[lazaronixon (_N_I_X_O_N_)]

That’s the point, I don’t know how to convince them. To be honest, I like to implement upload like Basecamp does, a single endpoint to upload files and use the signed_id to attach the file to records.

[brenogazzola]

Thanks!

Now that you mentioned it, this was actually a blind spot for me. I use Cloudflare, so my entire perspective on CDNs is of something that sits between my servers and my users, and therefore just using the proxy mode is enough to solve the CDN/caching issue. I didn’t even know the AST guide had that section

A couple of things of the top of my head I think could be done in AST:

1. The on demand controller should not process the variant if it not exists yet. Instead it should redirect to the original image (or a blank image) and enqueue a job to process the variant. By doing this you don’t risk image processing gobbling all memory/CPU that should be going to serve requests

2. The base representations controller needs an easy way to plug into its behavior so that you can modify the transformation hash before its passed to image processing. We’ve had to resort to monkey patching it because google had cached a jfif transformation that libvips could not handle and the error was constantly poping in Sentry.

3. A configuration option to place variants in a separate bucket or at least in a folder inside the ASTs bucket. This would make migrating between storage services easier. Even if you are not migrating, there’s been plenty of times I wished I could truncate the variants table and purge the files because the UX team decided to change the size of all images and the current variants are no longer needed.

And another that in theory can be handled by a good CDN, but would still help some people:

4. The default image analyzers should detect opacity, and use JPEG instead of PNG as a fallback if the image is opaque.

For item 1, there’s a recent PR on preprocessing variants, but that assumes that said variant will always be needed. As I’ve mentioned in my first post we don’t use has_many_attached and instead have a Photo model, which every other model users. Just because one model needs a 414x version of an image it does not mean that another does too.

Another problem with that approach is that if you are attaching a large amount of images (we’ve recently added a new vendor and had to attach over a million product images), you place a massive amount of load on your workers, all at once.

[Niklas]

Amazing post, thank you.

What would be your suggestion for improving the SetCookie header footgun your describing for Proxy mode?It’s still not good for setups like Heroku, where no proxy is available to drop the SetCookie header, isn’t it?

[brenogazzola]

Oh yeah, forgot about that one. I swear I remember a PR about this in the past, but can’t find it.

5. The ActiveStorage::BaseController should not send cookies along with the response. This causes CDNs that work like Cloudflare to bypass caching for these images, requiring extra configuration, and is a massive footgun for people who are configuring object caching in nginx and are not aware that the Set-Cookie header has to be removed before caching.

We were massively lucky with that one. It was already merged, just waiting for the CI queue to clear, when one of the helpdesk people who was testing something else in the QA server noticed they were being logged to a different account when he opened a specific product page.

[brenogazzola]

On Heroku you can’t use nginx, so that’s not much of a problem. And Cloudflare strips the set-cookie header before caching when you force it to.

[Spike]

I’ve been using activestorage-validator for the models that I own. I can’t off the top of my head remember if it works with direct uploads but my intuition tells me it probably won’t. It may just refuse to link the blob to the host model as an attachment which would leave you with a hanging blob that would get garbage collected eventually and wouldn’t be accessible. Again, that’s just my intuition.

[nickweiland]

Great post, thank you! Does anyone here know the correct Alpine Linux packages to use when moving from imagemagick?

[Envek (Andrey Novikov)]

As already mentioned above, ActiveStorage is great for uploading, and bad for variants generation and serving them to users: one have to install libraries to servers, run background jobs, store these variants.

And, as already mentioned above, there are image processing servers existing in the wild, both self-hosted like imgproxy, imaginary, thumbor, or cloud services like cloudinary. Dozens of them!

So I want to emphasize that you don’t need to care about all these processing yourself:

1. Use direct upload from client to storage (with help from ActiveStorage)
2. Generate variant URLs that point to caching proxy or CDN in front of image processing server
3. Let image processing server to generate variant for you (don’t store!)
4. and let CDN to cache it

There are a lot of upsides:

1. Performance: these things are just faster. Pipelines optimized for concurrent processing images, libraries are already initialized and loaded, etc. CDN cache misses isn’t that scary anymore. And application and image processing can be scaled independently.
2. Stability: Requests for images don’t interfere with requests to application server. They handle image compression bombs (and even if not, they will crash independently)
3. Less application code: no background jobs, no imagemagick/libvips on servers/containers, etc. Application becomes much simpler (in exchange for slightly more complex overall infrastructure).

This blog post writes about Rails app migration from “imagemagick on server” to “dedicated image processing server” approach so good: Imgproxy is Amazing

And I personally would like to recommend imgproxy (it is open-source and self-hosted). Partially because it has its own official Ruby client library for URL generation that integrates with Rails and has ActiveStorage support. And partially because I know folks who created it (my colleagues) and had so much joy using it. (so yes, I’m biased)