Add additional in-project directory to load custom Rails Command within the app

Hello,

Currently, in our project, we’ve migrated away from using Rake tasks and trying to use Rails Command classes instead.

However, one of the thing we started to notice is that we have to add our application’s lib directory to $LOAD_PATH in order for Rails to find the custom commands.

As I look into the source code, I noticed that Rails have a really specific rule on where to find additional commands:

https://github.com/rails/rails/blob/5371d5dcf26f5a5321471517531f36defa114e68/railties/lib/rails/command/behavior.rb#L55-L65

      # This will try to load any command in the load path to show in help.
      def lookup!
        $LOAD_PATH.each do |base|
          Dir[File.join(base, *file_lookup_paths)].each do |path|
            path = path.sub("#{base}/", "")
            require path
          rescue Exception
            # No problem
          end
        end
      end

``

https://github.com/rails/rails/blob/5371d5dcf26f5a5321471517531f36defa114e68/railties/lib/rails/command.rb#L104-L110

    def lookup_paths # :doc:
      @lookup_paths ||= %w( rails/commands commands )
    end

    def file_lookup_paths # :doc:
      @file_lookup_paths ||= [ "{#{lookup_paths.join(',')}}", "**", "*_command.rb" ]
    end

``

Basically, Ralis will only loop through every rails/commands or commands within the $LOAD_PATH. While this work perfectly for gems (as they can just have lib/rails/commands and everything will just work™) it might lead into a weird directory structure when you try to implement a custom command within the application itself, and generally I believe there’s a reason we don’t have our application’s lib directory in the $LOAD_PATH by default makes me feel wrong to add it back again.

So, I would like to propose a standard, well-documented path for custom command within the Rails application. Some candidates are:

  • app/commands — (This might not work well due to everything in app/ is autoloadable by zeitwerk)
  • lib/commands — (Goes along with lib/tasks, so might be a good choice)
  • config/commands — (A bit weird since this isn’t really a config) Do you think we should add one of these example to Rails lookup path? I’ll be willing to submit a PR to get it work.

Thank you,

Prem

Hey Prem,

The Rails command infrastructure is still private and I don’t like the current internal API that much. Don’t feel good exposing that. There’s also no way to mark command dependencies yet.

Now I did write the initial infrastructure 3-4 years ago for Rails 5.0, so it’s been some time coming. I remember fighting Thor a lot just to get to where it is today, now I’d like to think I’ve improved a lot in those years, but I’m still not looking super forward to going back in.

We could try to add this in an attempt to commit, but then again there’s really no rush.

So I’m not super keen, but I’m also not a complete no. Though I guess I’m saying if the proposal is just open as-is, add documentation, call it done, then it’s no from me.

Hey Prem! The lib directory is indeed in $LOAD_PATH, maybe your app does not have it for some reason?

Nope,

because code loader (Zeitwerk) isn’t setup on Rails commands,

it’s only load config/boot.rb (see https://github.com/rails/rails/blob/master/railties/lib/rails/generators/rails/app/templates/bin/rails.tt#L2),

and the boot.rb only load gems and Bootsnap (see https://github.com/rails/rails/blob/master/railties/lib/rails/generators/rails/app/templates/config/boot.rb.tt),

BTW, me +1 for this proposal, here’s my hacking https://github.com/jasl/cybros_core/blob/master/bin/rails

在 2020年1月15日星期三 UTC+8上午5:35:25,Xavier Noria写道:

Ah, I see now what is the point, commands are able to boot the application (eg, runner), but the lookup happens before.

I would not recommend app for that, because by default any subdirectory of app is in the autoload paths and eager loaded if eager loading is enabled.

Xavier

PS: BTW, Zeitwerk has nothing to do with $LOAD_PATH, the variable is modified by railties code as part of the boot process.

Yep, as 姜军 said — at the point in time where Rails tries to load all available commands (in boot.rb) the {project_root}/lib directory isn’t included in $LOAD_PATH yet.

I can confirm that after the app boots, the lib folder is indeed in the $LOAD_PATH. That definitely means lib/commands and lib/rails_commands are indeed the correct place to put the commands.

In that case, do you think it’d be ok for me to submit a patch to add {project_root}/lib to $LOAD_PATH before the command lookup happens?

After that, Kasper, I’ll dig through the history (Basecamp, GitHub, etc) to see where you left them off, figuring out what’s missing for it to be a full-fledged Rails feature, and give it another shot?

-Prem

That’s really a good feature, if you do that, we can write Rails command instead of Rake task.

BTW. {project_root}/lib/commands is a good place, because it has {project_root}/lib/tasks there and do the similar job.