Custom classes & Zeitwerk expectations

I needed to add a new service class that extended the ActiveStorage S3Service class, convention seems to promote putting the custom class in lib/active_storage/service/my_custom_service.rb, which also means the class namespace looks like ActiveStorage::Service::MyCustomService. To get this file to load properly (in development) I had to augment the eager_load_paths like so:

config.eager_load_paths << Rails.root.join('lib', 'active_storage')

This works fine, but is it correct? When this code was deployed to production, Zeitwork complained that my file my_custom_service.rb failed to define the const Service::S3WithPrefixService as expected. The inelegant solution is to drop this hack at the top of the file

module Service::S3WithPrefixService; end;

But what’s the Rails-way to add custom ActiveStorage services?

2 Likes

I do not have much experience with Active Storage, but from what I understand, Zeitwerk should not be involved. Let me explain.

When you configure a service, this code is invoked. As you see, a service configurator is instantiated, and that object ends up executing a require call for a file that has a certain expected path, followed by a const_get:

require "active_storage/service/#{class_name.to_s.underscore}_service"
ActiveStorage::Service.const_get(:"#{class_name.camelize}Service")

Therefore, no autoloading or reloading is going on here. If you define the service in

lib/active_storage/service/my_custom_service.rb

you do not need to do anything else, because lib is in $LOAD_PATH and Active Storage will be able to require the file. In particular, no need to configure eager load paths for this.

(Written off the top of my head by reading source code, please tell me if I got anything wrong.)

@fxn that is not my experience on Rails 6.0.2.2. With the custom service living in lib/active_storage/service/my_custom_service.rb and the body of that code looking like:

module ActiveStorage
  class Service::MyCustomService < Service::S3Service
  end
end

When I attempt to use this service, I get the following error from Zeitwerk

Zeitwerk::NameError: expected file /app/lib/active_storage/service/my_custom_service.rb to define constant MyCustomService, but didn't

Do you have the following file under app? /app/lib/active_storage/service/my_custom_service.rb

No, it’s in the top-level lib directory, which is where all the online tutorials & guides I’ve read recommend placing custom services.

The Rails guides themselves on this subject are pretty sparse, so it leaves us devs up to go figure it out for ourselves where to put our services

Might help to look at how it’s done in the Cloudinary gem?

@justinperkins have you deleted this?

config.eager_load_paths << Rails.root.join('lib', 'active_storage')

Let me explain what happens in other words.

First, since Active Storage performs a require and const_get by itself (as shown above), your file is correctly located, because lib belongs to $LOAD_PATH.

You do not need to configure eager_load_paths for this to work, it just works.

Not only that, and that is my point, the configuration

config.eager_load_paths << Rails.root.join('lib', 'active_storage')

has to be deleted.

That is telling Rails that lib/active_storage is an autoload path, and so that directory acts as a root directory, so it corresponds to Object (like app/models does). That is why Zeitwerk gets involved, and why it complains (because if you take that directory as Object, the file does not follow the conventions).

Bottom line: delete that configuration and your application should be good.

2 Likes

@fxn Yeah I deleted that line per your recommendation.

To summarize, with a custom service called MyCustomService here: RAILS_ROOT/lib/active_storage/service/my_custom_service.rb and a body of:

module ActiveStorage
  class Service::MyCustomService < Service::S3Service
  end
end

I get the following error when attempting to load the class ActiveStorage::Service::MyCustomService:

Zeitwerk::NameError: expected file /app/lib/active_storage/service/my_custom_service.rb to define constant MyCustomService, but didn't

Adding this to the end of the file resolves the issue and allows me to use my custom service

class S3WithPrefixService < ActiveStorage::Service::S3WithPrefixService; end

@justinperkins I believe there is something missing in the situation.

If Zeitwerk is looking into lib, it means it has been configured to look into lib, which is not by default.

Not only that, if it believes that lib/active_storage/service/my_custom_service.rb should define MyCustomService (note there is no namespace), that means that lib/active_storage/service is in the autoload paths.

Could you please run

$ rails runner 'p ActiveSupport::Dependencies.autoload_paths'

and also throw

Rails.autoloaders.log!

in config/application.rb after loading the framework defaults, trigger the error, and share the traces?

1 Like

Here you go

root@211287ca9557:/app# rails runner 'p ActiveSupport::Dependencies.autoload_paths'
Running via Spring preloader in process 52
["/app/app/channels", "/app/app/controllers", 
"/app/app/controllers/concerns", "/app/app/helpers", 
"/app/app/jobs", "/app/app/mailers", "/app/app/models", 
"/app/app/models/concerns", "/app/app/services", 
"/app/app/services/concerns", 
"/bundle/vendor/ruby/2.7.0/gems/bootstrap_form-4.5.0/lib/bootstrap_form/lib", "/bundle/vendor/ruby/2.7.0/gems/devise-4.7.1/app/controllers", 
"/bundle/vendor/ruby/2.7.0/gems/devise-4.7.1/app/helpers", 
"/bundle/vendor/ruby/2.7.0/gems/devise-4.7.1/app/mailers", 
"/bundle/vendor/ruby/2.7.0/gems/actiontext-6.0.2.2/app/helpers", 
"/bundle/vendor/ruby/2.7.0/gems/actiontext-6.0.2.2/app/models", 
"/bundle/vendor/ruby/2.7.0/gems/actionmailbox-6.0.2.2/app/controllers", 
"/bundle/vendor/ruby/2.7.0/gems/actionmailbox-6.0.2.2/app/jobs", 
"/bundle/vendor/ruby/2.7.0/gems/actionmailbox-6.0.2.2/app/models", 
"/bundle/vendor/ruby/2.7.0/gems/activestorage-6.0.2.2/app/controllers", 
"/bundle/vendor/ruby/2.7.0/gems/activestorage-6.0.2.2/app/controllers/concerns", 
"/bundle/vendor/ruby/2.7.0/gems/activestorage-6.0.2.2/app/jobs", 
"/bundle/vendor/ruby/2.7.0/gems/activestorage-6.0.2.2/app/models",
"/app/test/mailers/previews"]

Regarding the autoloaders log, I’d prefer not to share those results publicly. I have the results saved and can send it, or look for something in particular if that helps. There’s nothing referencing /lib in the results

That does not square.

Just to make sure, could you please restart Spring? Not saying it is related, but since things are working in a way that does not square, I’d like to try this just in case.

If you reproduce after restarting spring (spring stop), sure, send the traces privately to fxn@hashref.com.

1 Like

The above trace was from a fresh launch of the docker instance, spring was ready because I fired up rails console by mistake before running the rails runner. I emailed over the output from Rails.autoloaders.log!

This has been understood via private emails. Let me followup here for the archives.

The eager load configuration somehow was still present, once you removed it the Zeitwerk error disappeared as expected. This was kinda a red herring. The error happened because the file did not follow the conventions, the root directory did not match namespaces.

Once the eager load configuration was removed, we could move on.

OK, who loads the configured service class? Active Storage does via a manual require + const_get.

Worth noting also, that if you try to reference your service class in a freshly fired console with eager loading disabled, it is not found (regular NameError). Reason is Active Storage is lazy loading the configured service. As you see in the code just linked, the service is not loaded until the builtin class ActiveStorage::Blob is loaded.

3 Likes

Yup, thanks a ton of the help @fxn

The NameError in a fresh console session sent me off on a wild goose chase that wasn’t even needed. Ended up cobbling together several snippets of superfluous code.

On the bright side, I have a better understanding of how it all works, especially the lazy loading

2 Likes