Initialization autoloaded the constants x, y, and z. Isn't enough information

Rails is amazing, I spend most of my professional life working in it, and love it. Thank you all!

The recent rails 6.0 upgrades introduced the Zeitwerk loader which has been mostly smooth for me across my many projects, with one exception.

The deprecation about autoloading constants during initialization doesn’t provide enough information to pinpoint the invocation that loaded the classes too early.

DEPRECATION WARNING: Initialization autoloaded the constants x, y, and z.

Being able to do this is deprecated. Autoloading during initialization is going to be an error condition in future versions of Rails.

Reloading does not reboot the application, and therefore code executed during initialization does not run again. So, if you reload ApplicationRecord, for example, the expected changes won’t be reflected in that stale Class object.

These autoloaded constants have been unloaded.

Please, check the “Autoloading and Reloading Constants” guide for solutions. (called from <top (required)> at …exampleapp/config/environment.rb:5)

The guide is great, but with large projects, for which I was not the only author and I don’t know every corner tracking down these invocations is still difficult. It would be tremendously helpful if there were any way to track an enumerate the specific invocations in the deprecation message.

6 Likes

It sounds like just adding something like:

Deprecated autoloads occurred in: 
./config/initializers/foo.rb:89, Bar
./config/initializers/example.rb:26, Baz

would do a lot to help you out, based on how you’re describing this problem.

@fxn How feasible is this proposal given Zeitwerk’s architecture? Are you already tracking which invocations caused a deprecated autoload?

2 Likes

Agreed, this should be better and this is a clear sign of a WTF because the key is hidden in here:

You’ll see that line because ActiveSupport::Deprecation via Rails.backtrace_cleaner has cleaned the backtrace of Rails, Ruby and gem references, leaving only your app code to show.

I think I’ve seen this deprecation in an app I’ve worked on and there it did show the correct line in the app, but my memory is spotty. The bottom line is that this is an issue with the deprecation handling we have and I’ve seen it strip useful information before. I think there’s a heuristic we can apply to help us detect pointing at an unhelpful line in the backtrace.

Your exampleapp is it on GitHub, so we can take a look at it? Since it looks like you’ve already reproduced this we could maybe start from there.

Kasper, Unfortunately the code is not on github, nor can it be.

I hesitate to get into too much detail here, esp because it is an open bug so I speak without fully understanding. If I knew enough to provide a toy example, I would. Here is some context of the specific case. The actual classes from the example were ApplicationRecord, AuthUser, and User. I’ve been through all the initializers and see no reference to any of those. My hunch is that one of our internal gems (a wrapper around auth_logic) itself is constantizing a yaml config of what the user model should be and storing that too early in the process. Point being the error may not be in this application’s code at all, but one of our gems. Even if it is a gem’s fault though, I’m left in the same situation of desiring more information.

Might the library also, or alternatively, allow a configuration to turn this deprecation warning into an exception now, so a full trace is produced? Or is there a way to turn the backtrace_cleaner off?

Hello @andy_nutter-upham.

Let me clarify first that this deprecation is unrelated to Zeitwerk.

Rails has an autoload logic, and it defers the technical details of how to autoload, how to reload, etc. to its autoloaders. Before Rails 6, and going back to 2004, there was only one choice. Nowadays you have two.

When things reload, how does the application boot and sets things up, how do you lock to make reloading thread-safe, etc. That is Rails-specific logic. For example, other web frameworks autoloading with Zeitwerk have their own logic.

I am not saying you had to know this, of course, just explaining it so that you can build a mental model.

OK, so you’ll get this warning because Rails issues it, and that happens in both autoloading modes. It is a new warning of Rails 6.

Once that precision has been made, unfortunately none of the autoloaders have information about where was the constant lookup triggered. Ruby does not give you that information. You know in which file is the constant defined, but not which reference to the constant caused the lookup.

However, since that happened in the initializers, the scope is reduced. Your first option is to grep them:

ag Foo config/initializers

If, for some reason that is not helpful, you can always go to the file foo.rb and put at the top

pp caller_locations

This last technique works in any scenario.

5 Likes

Let me complement the intro to reinforce the mental model.

In this particular case, the problem is that Rails does not run the initializers when it reloads the application code and that does not work well with reloading (see why here). That is Rails logic, independent of the autoloaders, see?

This has been a historically unaddressed peril that was only documented. However, software does not have to document gotchas, you have to make gotchas not possible. And Rails 6 has finally made something about it, first with a warning because we try to not break updates as much as possible, and eventually it will become an error condition.

The way to address proper autoloading gotchas has been to write Zeitwerk, all the known ones are gone. And that is why zeitwerk mode will eventually be the only option. Right now, we have 16 years of applications using classic and have to be extremely careful in the transition.

Xavier, Thank you so much!!!

All excellent clarifications. I had conflated the new deprecation warning with the new autoloader.

I had already done the first search you suggested, but since it was an issue coming from the gem, nothing turned up. Your final suggestion of made the cause obvious! Adding pp caller_locations to the User model pointed me directly at the issue in our gem.

The cause was the engine doing this:

  initializer 'InternalGemName User Model' do |app|                                    
    ActiveSupport.on_load(:active_record) do                                    
      InternalGemName.user_model ||= User if File.exist?("#{Rails.root}/app/models/user.rb")
    end                                                                       
  end 

I can change this around so the InternalGemName.user_model’s first invocation can do this auto-detection later. Which fixes the issue by removing this initializer in its entirety. Awesome!

But, if you could clarify a little further, I’d love to understand why the on_load deferral didn’t help. I’m surprised that the User record class was captured in an undesirable way given that it is in a block that shouldn’t get executed until, presumably after it is ok? Moreover changing User to “User”.constantize didn’t help at all. I’m still not understanding something important here.

Or more generally, if you need to configure a gem to point at a specific class by default. How would you go about that / or when?

ActiveSupport.on_load(:active_record) triggers very early in the boot process. If you look here you can see it. That gets triggered as soon as active_record/base is first required, which will happen in config/application.rb I believe.

Since your code looks like it’s related to business logic rather than framework things, I think you could potentially move that callback later in the process.

Does this work?

config.after_initalize do
  InternalGemName.user_model ||= User if File.exist?("#{Rails.root}/app/models/user.rb")
end

Within the same engine class as the initializer code.

1 Like

Ah! That hook loads much earlier than I’d realized.

config.after_initialize works great too. (Update: apparently not, see Xavier’s response below)

Thank you all. Unfortunate that the message can’t be made more explicit for future users, but the explanations here are a resource in themselves!

However, the problem still exists. Initializers run once, and InternalGemName.user_model is caching an autoloaded class object. Therefore, if you reload, that class object becomes stale.

Let me show you with a global variable:

Rails.application.config.after_initialize do
  $USER = User
end

then fire a console.

When you inspect the class stored in the global variable and the one stored in the constant, they are the same object:

> $USER.object_id
=> 70197445851400
> User.object_id
=> 70197445851400

Now reload:

> reload!

and check again:

> $USER.object_id
=> 70197445851400
> User.object_id
=> 70197429247480

See? User got a new class object (was reloaded). The global variable has now a stale object. That means that if you changed the implementation of the User class, the engine won’t see the changes.

This is a consequence of the extremely flexible and decoupled Ruby model for this: Class and module objects are not special, they are regular objects. On the other hand, variables and constants are not special, they are just storage. Constants belong to modules, quite literally. You have three pieces of the Ruby model playing together, and they are free of each other. You can move class objects around, you can store them anywhere, you can delete constants. Also, files can define an arbitrary number of classes, and they can have arbitray names, and live in arbitrary namespaces. It’s all very decoupled!

That is why Zeitwerk defines strict project constraints, to make the problem solvable. There’s API to ignore, to inflect, to collapse, etc., the API of the library itself has some flexibility, but some fundamental assumptions are required. (The Rails integration does not expose them because it has to provide the same API of classic, which does not have them.)

Back to this post. It’s the same thing again: Reloading does not run the initializers. Any.

The proper solution for this use case is to put that code in a reloader callback. For example:

Rails.application.reloader.to_prepare do
  $USER = User
end

That is called once at boot, and then every time you reload (twice for historical and obscure reasons, make it idempotent).

Would be handy to have a hint like this in the warning message probably. I’ll add something like this to the section about initializers in the guide too.

4 Likes

This reply helped me a lot. I am wondering if it could be a good idea to write into the documentation.

I know this quite “hacky” to tell the developer to add a puts caller on the top of the class to identify but it helps. :blush:

For me adding:

puts caller_locations.select { |line| line.to_s =~ /#{Rails.root.to_s}\/config\/initializers/ }

just after constants mentioned into the deprecation warning, helped me to identify quickly the origin.

5 Likes

This did the trick for me ! I was able to identify who was causing the warning :wink: Thanks.