Rake tasks in production, application models, and Rails 2.3.8-3.0

rubyonrails-core
1

Consider this simple rake task:

task :heavy_process => :environment do

MyModel.process

end

The problem I’m seeing here is that this will work in development mode, in which autoloading kicks in when the “MyModel” constant is referenced for the first time, but will fail in production mode because of the behavior described below. This issue has been bugging me for a while. I can’t recall it being a problem with Rails 2.3.5. Not sure if this is expected behavior.

If someone is writing a rake task that needs the application to be loaded, he or she will make it depend on the “:environment” task which will boot the app in the respective environment. It will also set the $rails_rake_task global to true. This has been around for a while.

However, since the 2.3.8 and 3.0 releases, this global prevents preloading of application classes in production (or any environment in which “cache_classes” setting is enabled). This is probably so the app boots faster; users are expected to require what they need in the rake task. Here is this logic in Rails 3:

initializer :eager_load! do

if config.cache_classes && !$rails_rake_task

ActiveSupport.run_load_hooks(:before_eager_load, self)

eager_load!

end

end

Going back to our initial rake task, our initial reaction is that we should put require my_model before trying to reference MyModel. However, if MyModel is linked to other models via associations, we also need to require all of the associated models in case the MyModel.process method uses these associations. In the end, wherever we put the require (or require_dependency) statements, it just feels wrong and clumsy.

We’ve also been seeing migrations fail in production mode because they reference a model to do some processing. Again, the biggest problem here is that all this works like a breeze in development, making us developers feel relaxed and secure, but then surprises us when we push to production.

It occurred to me that an easy solution might be to turn off “cache_classes” setting in production if $rails_rake_task is true. Comments on that?

Thanks

2

. In the end, wherever we put the require (or `require_dependency`) statements, it just feels wrong and clumsy. We've also been seeing migrations fail in production mode because they reference a model to do some processing. Again, the biggest problem here is that all this works like a breeze in development, making us developers feel relaxed and secure, but then surprises us when we push to production. It occurred to me that an easy solution might be to turn off "cache_classes" setting in production if $rails_rake_task is true. Comments on that?

eager loading is a relatively new piece of functionality, and disabling it for rake tasks should *ideally* simply fall back on the const_missing hook. I'm a little confused why it's not doing that. Do you have config.threadsafe! set or something?

Either way, unless you have config.threadsafe! set, this is *clearly* a bug. And if you do, we should probably force the eager loading to take place irrespective of whether it's a rake task.

3

You’re right, we had threadsafe on. I still think it’s a bug, because rake tasks are not threaded web server runtime and therefore shouldn’t guard against race conditions and such.

For now I’ll just turn off threadsafe (I guess it’s not needed for Passenger?). Thanks Koz

4

Passenger should be able to make use of threadsafe mode. Either way, the problem here seems to be that Rake tasks have a kludge in this one area of the system, but not in others. We either need to decide that Rake tasks are never considered threadsafe (even if the threadsafe! flag is on), or eager load for Rake tasks.

I’m surprised nobody else hit this before.

Yehuda Katz Architect | Engine Yard (ph) 718.877.1325

5

Actually, this is happening in my project since the beggining. I just didn’t have enough time to debug this behavior. I required the necessary files from inside the task to solve my needs…

Rodrigo.

6

You're right, we had threadsafe on. I still think it's a bug, because rake tasks are not threaded web server runtime and therefore shouldn't guard against race conditions and such. For now I'll just turn off threadsafe (I guess it's not needed for Passenger?). Thanks Koz

config.threadsafe! just does:

    # Enable threaded mode. Allows concurrent requests to controller actions and     # multiple database connections. Also disables automatic dependency loading     # after boot, and disables reloading code on every request, as these are     # fundamentally incompatible with thread safety.     def threadsafe!       self.preload_frameworks = true       self.cache_classes = true       self.dependency_loading = false       self.action_controller.allow_concurrency = true       self     end

It's the dependency_loading = false that's messing you up. The initializer should probably be checking that, not cache_classes.

7

How about changing that line to:

self.dependency_loading = defined?($rails_rake_task) && !!$rails_rake_task

Good idea?

8

How about changing that line to: self.dependency_loading = defined?($rails_rake_task) && !!$rails_rake_task Good idea?

That has a few potential side effects, but how does this look for the 2-3-stable change:

9

Looks good. Why just 2.3-stable?

10

Looks good. Why just 2.3-stable?

2.3 needs a 'smallest possible bugfix' change, master can 'do the right thing' which probably *isn't* adding another conditional.