Active Storage S3 Expied images

[rctneil (Neil Tonge)]

Hi,

I have an app i’m working on and in Production it uses Amazon S3 to store images.

The images show up fine but when we come back to the app after leaving it open in a tab for a short time, we do a regular refresh and the images fail to load. If I try to open the image in a new tab I see:

<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<X-Amz-Expires>300</X-Amz-Expires>
<Expires>2025-02-06T10:00:24Z</Expires>
<ServerTime>2025-02-06T14:57:20Z</ServerTime>
<RequestId>G2QV025B5WWFA6dEM</RequestId>
<HostId>iaQhWt94gsss1xHt3k2pLefeeeh6R3QUbDLn+2Emkuucf9fz48O739VoUTM08Sq/Fvudfi24nqKd7yAFA/4MUa5yp7rwVpjlBpuOdOJjd93</HostId>
</Error>

If I do a hard refresh, the image typically then reappears.

The Active Storage guides talks about Public and Private access and how private access generates signed single use URL’s. I’m assuming that this is the case in this app.

How do I choose whether to use Public or Private?

If my images are showing, why does a single refresh not generate a new signed single use URL for that refresh?

I’m really confused. Can anyone shed some light on this for me and how this all works?

Thanks

[brenogazzola]

If you are going to use Active Storage in production, I recommend checking my guide here in the forums, since it’s a through explanation of how it works an all the problems you might face in production.

Section 2.3.3 is where I explain the three modes of operation: redirect (default, which you are probably using), proxy (supports CDN caching), and public (no CDN cache, but simplest to use)

If you decide to use public mode, check this section of the Rails guides for how to do it, and use then change this:

<%= link_to user.avatar %>

To this

<%= link_to user.avatar.url %>

[rctneil (Neil Tonge)]

@brenogazzola

Ok, Thanks for that, super article! Could you please explain though, how should I determine when to use redirect mode and when to use public mode?

I’m very confused.

[brenogazzola]

Active Storage was initially built by 37Signals for Basecamp, their project management app. Since users in a company were not supposed to view files of other users in another company, and many of these files were sensitive, they built Active Storage to generate temporary links when the page containing those files was opened. This was done through “redirect mode”.

This mode can be used for images, as long as you are only using them in your own website, and not using fragment caching in the views that contain those URLs. If you need to send them through an email, slack message, or anything similar, redirect mode won’t work since the link will expire after 5 minutes. Redirect mode also does not work with CDN caching, since the URL is not a file, but another URL.

In Rails 6 this was changed with proxy mode, where Rails streams the file from storage. This keeps the main advantage of Active Storage, which is on-the-fly image transformations (compress, crop, resize, etc), but increases load on the servers since you have puma workers stuck streaming the data. This is the one I use, but I keep nginx in front of Puma, with object caching and buffering enabled, so each file is served only once, which solves the extra load problem.

Finally, public mode was added to avoid the whole “urls expires” and “files need streaming”. The problem of this mode, is that if you need a image variant, you need to process it before generating the url to put in the HTML. So if you have a feature where a user uploads multiple images (let’s say 5), and in the next page you show thumbnails of these images, your users will spend a long time looking at a blank page while active storage processes those 5 images to generate the urls.

TL:DR:

1. Use redirect mode if you images are only visible in your web app (no emails, no slack/whatsapp/discord, etc)
2. Use proxy mode if you have a proxy type CDN (Cloudflare) so they can be cached.
3. Use public mode if you need images to be visible outside your app.

You can actually use all three modes at the same time. In your case, the simplest would be stay in redirect mode, and when necessary, use the public url of the image.

So in your views:

<%= image_tag user.avatar.variant(resize: [100,0] %>

And in your emails:

<%= image_tag user.avatar.variant(resize: [100, 0]).processed.url

Just make sure your storage.yml file has public: true in the config for S3/R2/etc.

[rctneil (Neil Tonge)]

Ok, that makes sense.

Your last line says " Just make sure your storage.yml file has public: true in the config for S3/R2/etc." - You mean only for public mode or for redirect mode too?

After reading your message I am happy to use redirect mode but wondering why when I do a page refresh the images fail to display.

I’m wondering if this is a cache thing and not an AST thing? Is there a way to make them only cache for the length of time that the signed one time use url is valid so a basic simple refresh will refetch them?

[brenogazzola]

Just for public. Redirect and Proxy work out of the box.

Is that local or production? It’s normal for redirect to fail in production if you have something caching the links (cloudflare/nginx) since they will cache the redirect link (which expires), and not the image

[rctneil (Neil Tonge)]

Ok,

Yes, in production this is happening. The link includes the AWZ expiry of 300 seconds so confident it’s a caching issue. How do I tell it to not cache for longer than that?

Hosting is a simple Digital Ocean VPS configured by Hatchbox.

Thanks

[brenogazzola]

As far as I remember, active storage already sends a no-cache header for files in redirect mode. So if it’s breaking, them your server is serving old urls, or something on the stack is ignoring in the no-cache rule. Unfortunately I can’t help with with that, you will have to investigate

[collimarco (Marco Colli)]

@brenogazzola

Finally, public mode was added to avoid the whole “urls expires” and “files need streaming”. The problem of this mode, is that if you need a image variant, you need to process it before generating the url to put in the HTML. So if you have a feature where a user uploads multiple images (let’s say 5), and in the next page you show thumbnails of these images, your users will spend a long time looking at a blank page while active storage processes those 5 images to generate the urls.

I think that this part of the explanation is not correct. The default is a “redirect” strategy even if you add public: true to a bucket in storage.yml. The only difference is that the Rails ActiveStorage controller redirects to a public S3 URL instead of redirecting to a signed S3 URL.

[rctneil (Neil Tonge)]

I’m still struggling massively with this. The images show up after a hard refresh, but when i then leave the tab and come back to it later and do a soft refresh they all vanish. Even navigating around the site doesn’t make them appear. (The site does use Hotwire).

The site is on a server configured by Hatchbox and is totally unchanged from the initial Hatchbox configuration. I checked with Hatchbox and they’ve said they don’t add or configure any caching stuff

Everything is configured in the most basic, default way possible. a basic default S3 bucket, a default simple AST setup and a default simple Hatchbox config.

I’m really running out of ideas as to what’s happening here? I’ve done what feels like endless research to try and find an answer but not finding anything.

[brenogazzola]

It still defaults to to redirect, but if you call .url on the file you get the public url instead of the redirect url