How about a pre-initialization hook in boot.rb?

rubyonrails-core
#1

Howdy,

Please take a look at Enhancement #9943 (2) and see what you think.
Here's the pitch for this little 5-line patch:

Even though boot.rb has the familiar "Don't change this file..."
message, sometimes I want to.

This would allow dynamic control over RAILS_GEM_VERSION and the
Version Specification passed to the RubyGems 'gem' method for the
rails gem. For example, users could test an app or plugin in a
Continuous Integration environment which performs automated regression
testing against various old Rails versions.

This patch would also to facilitate my GemInstaller gem, which
centrally manages the installation and loading of all gem dependencies
and versions, including Rails. Currently, I have to tell people (1)
to ignore the dire "Don't change this file..." warning and hack
boot.rb if they want to manage the Rails gem version along with the
rest of their gems. This is not compatible with Rails upgrades which
modify boot.rb, such as changeset [7832] and ticket #9834; users will
have to re-modify boot.rb after upgrading.

So, whaddaya think? I'm happy to rewrite this patch however you would
like. I thought the simplest and most flexible approach would be to
mirror the config/initializers directory approach, which is done in
just 5 lines. If you don't like the extra config/preinitializers
directory, it could just look for a single hook file like
config/preinitializer.rb. This would be one less default dir in
/config, but it would be less flexible. For example, I couldn't just
have GemInstaller dump a provided geminstaller_preinitilizer.rb file
into config/preinitializers, I'd have to modify the single hook file.
If it's OK to have an extra dir in config, though, I think the current
approach is best

One other note. This patch is untested, because boot.rb would require
some refactoring to be testable, and I didn't want to overengineer
this patch. For an example of how boot.rb could be more made more
testable, see #9834. We could even pull out all of the logic from
boot.rb into a class, which would be much easier to test in isolation.
I'm definitely up for this if you are interested, but I wanted to do
The Simplest Thing That Could Possibly Work as the first cut.

Thanks,
-- Chad Woolley

(1) http://geminstaller.rubyforge.org/documentation/tutorials.html#integrating_geminstaller_into_ruby_on_rails
(2) http://dev.rubyonrails.org/ticket/9943

#2

This would allow dynamic control over RAILS_GEM_VERSION and the
Version Specification passed to the RubyGems 'gem' method for the
rails gem. For example, users could test an app or plugin in a
Continuous Integration environment which performs automated regression
testing against various old Rails versions.

I'm just not sure that there are a whole lot of use cases that need
this kind of flexibility before environment.rb gets hit.

When I'm testing my apps against multiple rails releases I just use
symlinks in /vendor/rails pointing to various subversion checkouts.

Are there any other common cases you have in mind? You could always
automate the patching of boot.rb for your gem installer app.

#3

> This would allow dynamic control over RAILS_GEM_VERSION and the
> Version Specification passed to the RubyGems 'gem' method for the
> rails gem. For example, users could test an app or plugin in a
> Continuous Integration environment which performs automated regression
> testing against various old Rails versions.

I'm just not sure that there are a whole lot of use cases that need
this kind of flexibility before environment.rb gets hit.

Hi Koz,

Thanks for the feedback! I'll respond to your points inline below.

When I'm testing my apps against multiple rails releases I just use
symlinks in /vendor/rails pointing to various subversion checkouts.

Symlinks aren't cross-platform. They are also harder to automate,
especially in a Continuous Integration environment. With a hook to
pre-initialize RAILS_GEM_PATH and load the gem on the path, you can
control everything with an environment variable and not modify the
filesystem/working copy at all - everything is in memory and the env
var goes away after the build. The symlink approach is harder to
automate and more intrusive - it would require the CI tool to manage
the physical creation of the symlinks (and ensuring the existence of
wherever they point to), and reseting them to the proper defaults (but
what if the build fails? Hope your Ci tool has a good
after-build-failure hook).

IMHO, automating the use of multiple releases is easier with gems than
modifying the filesystem under vendor, if only because of RubyGems
built-in support for selecting any installed version on the fly.

Are there any other common cases you have in mind? You could always
automate the patching of boot.rb for your gem installer app.

Well, that would be ignoring the warning to not modify boot.rb, and I
don't want to do that!

Also, that would involve some ugly parsing and hacking of people's
boot.rb (the hook needs to be after RAILS_ROOT is defined). What if
boot.rb changes with a new Rails release (or has been customized by
the user) and my regexp no longer matches? What if I have a bug and
hose their boot.rb?

Again, I think it's cleaner, safer and less intrusive to have
preinitializer hook(s). No need to ignore warnings and modify
boot.rb, and if I screw something up or users don't want my hook
anymore, there's a well-defined place to find and delete it.

Even if there's not a ton of use cases for this patch, there's not
really any alternatives that address the problem well, and it's
completely innocuous AFAIK, so I hope it can be considered in that
light.

--
Cheers

Koz

Thanks again for the response,
-- Chad

#4

I would love to see this patch (or a similar one) make it in, as we
are dealing with this very issue with a tool we built test our apps
and plugins against multiple versions of Rails in our CI environment.
We went with the rubygem approach because it lets use gems for version
management, but we are hitting issues in apps due to the way the boot
process works.

thanks,
Rob

http://robsanheim.com
http://thinkrelevance.com
http://streamlinedframework.org

#5

I would love to see this patch (or a similar one) make it in, as we
are dealing with this very issue with a tool we built test our apps
and plugins against multiple versions of Rails in our CI environment.
We went with the rubygem approach because it lets use gems for version
management, but we are hitting issues in apps due to the way the boot
process works.

What are the other issues that you've hit? If it's just about choosing
a different version of rails, then can't you just set
RAILS_GEM_VERSION in environment.rb?

I still don't buy the need for a pre-initializers rig, but perhaps
there's a simpler option hiding somewhere.

#6

Nope, this doesn't work. boot.rb and initilization always hits before
environment.rb is interpreted, which means any code in environment.rb
is too late to have control over the rails version. See my comments
on the patch.

By the way, I am working with this approach and the latest boot.rb
code in my projects today, and it's working great. All I have to do
is set the RAILS_GEM_VERSION environment variable, and that
automatically loads the correct rails gem version.

I also have this hooked into cruisecontrol.rb, so that it
automatically sets the rails version based on the name of my project -
e.g.: a project name of myproject_rails_1.2.5.7919 automatically uses
that gem for rails. This makes is a no-brainer to test my project
under multiple rails versions via CI, and requires no code changes at
all - they all use the same subversion url.

Thx,
-- Chad

#7

I just saw this was taken off the 2.0 milestone list. This is a
simple patch with tests which addresses a real limitation of the
current Rails. I don't think I've done a good job explaining it so
far, so I'll try again in the form of a failing test scenario.

Here is the scenario which is not currently possible without a
preinitializer hook:

1. I want my app to use the stable gem (e.g. 1.2.5) by default, _even_
if I have a beta gem installed (e.g. 1.2.5.7919)
2. I also want to use RubyGems Advanced Versioning [1] in my
RAILS_GEM_VERSION environment variable to use the _latest_ beta gem
that I have installed, e.g.: RAILS_GEM_VERSION='>1.2.5' or '~>1.2.5.0'
3. I want this to happen automatically without having to modify my app
(no freezing rails or modifying environment.rb), because I want my
Continuous Integration system to automatically run my app against
multiple rails versions for regression and edge-rails testing by
simply setting the RAILS_GEM_VERSION env var prior to the build.

With this patch, I can have complete control over the gem version
which is loaded, and what is set in the RAILS_GEM_VERSION constant
before the normal boot gem-selection process occurs.

I'm flexible, if you want me to rewrite it to not have the
'preinitializers' directory and instead just have a magic hook
filename, I'm glad to rewrite the patch. Just let me know.

Thx,
--Chad

[1] http://www.rubygems.org/read/chapter/16#page76

#8

If all you want to do is change RAILS_GEM_VERSION to test different
versions of Rails, why don't you just change boot.rb to default to
ENV['RAILS_GEM_VERSION']:

Index: config/boot.rb

#9

The boot.rb has been completely refactored on trunk (and improved,
thanks bitsweat!). Your patch is against an old version.

Second, the version specification in the 'gem' method in boot.rb is
hardcoded to "equal", even in the new boot.rb. This doesn't meet my
second requirement - being able to pass a RAILS_GEM_VERSION like
'>1.2.5' in the env which automatically uses the latest beta gem
without hardcoding the version.

Thanks for the reply, though!
-- Chad

#10

This would also mean hacking boot.rb, which is not a very nice
solution and means additional work merging everytime you upgrade
rails.

- Rob

#11

I'm flexible, if you want me to rewrite it to not have the
'preinitializers' directory and instead just have a magic hook
filename, I'm glad to rewrite the patch. Just let me know.

My problem is that to add an entire pre-initializer feature when the
sole use case is version twiddling seems hugely overkill. No one's
come up with a case that can't be solved by moving things into and out
of vendor/rails...

If all you want to do is have more control over the version of rails
that gets required, let's do that instead of some generic thing that
we'll have to support for an entire release cycle.

#12

This would also mean hacking boot.rb, which is not a very nice
solution and means additional work merging everytime you upgrade
rails.

Obviously any change to boot.rb that isn't included in Rails will
require merging on upgrade. In case it wasn't clear, I was suggesting
the change (or a similar one) be included with Rails. Rails uses
ENV['RAILS_ENV'] for the environment if available, so using
ENV['RAILS_GEM_VERSION'] for the gem version is similar in that
regard. It solves this particular problem and I can't see any
downsides.

Jeremy

#13

> I'm flexible, if you want me to rewrite it to not have the
> 'preinitializers' directory and instead just have a magic hook
> filename, I'm glad to rewrite the patch. Just let me know.

My problem is that to add an entire pre-initializer feature when the
sole use case is version twiddling seems hugely overkill. No one's
come up with a case that can't be solved by moving things into and out
of vendor/rails...

The use case is selecting the version _without_ modifying the app. I
want _easily_ test my app against multiple rails versions under
Continuous Integration. Having my CI tool simply set an environment
var is easy. Having it install plugins or hack externals is much
harder, relatively.

If all you want to do is have more control over the version of rails
that gets required, let's do that

Sure, we could have the RAILS_GEM_VERSION allow a version spec (>)
instead of just a numeric version. However, this still wouldn't
support even cooler preinit hooks that I haven't even mentioned to
avoid confusion (like automatically installing the required gem if
it's missing)

... instead of some generic thing that
we'll have to support for an entire release cycle.

What's to support? It will be completely transparent to people who
don't use it. If it's the existence of the directory that's a
problem, I'm fine to go with the single magic hook file (e.g.,
config/preinitialize.rb)

--
Cheers

Koz

Thanks for the feedback,

-- Chad

#14

The use case is selecting the version _without_ modifying the app. I
want _easily_ test my app against multiple rails versions under
Continuous Integration. Having my CI tool simply set an environment
var is easy. Having it install plugins or hack externals is much
harder, relatively.

Again, you're completely missing the point of what I'm objecting to.
If all you want to do is change the gem version, lets change boot.rb
to check an env var first. There's yet to be a single use case for
anything else.

Jeremy's suggestion looks like the right way forward, so lets stop
wasting time talking past one another or trying to design some generic
'pre boot' hooks, and instead apply a patch that lets you solve the
problem you have.

#15

The latest boot.rb in trunk does allow override by the env var.
However, this still only supports a specific version, not an advanced
version spec (e.g. floating on latest beta gem if env exists).

Also, as I said in the other thread, even if it the env var did take a
version spec instead of a specific version, it still wouldn't allow
cooler advanced hooks like auto-installation of missing gem versions
on app startup.

We are an agile shop that works on a lot of different rails projects,
so we switch pairs, machines and projects often. These are often on
different rails releases, which may or may not be installed on a given
machine. We also want to have our CI systems automate testing of all
our projects against multiple rails versions, including the latest
edge gems, without having to even think about it.

It's all about the automation :slight_smile:

Thanks,
Chad

#16

> Obviously any change to boot.rb that isn't included in Rails will
> require merging on upgrade. In case it wasn't clear, I was suggesting
> the change (or a similar one) be included with Rails. Rails uses
> ENV['RAILS_ENV'] for the environment if available, so using
> ENV['RAILS_GEM_VERSION'] for the gem version is similar in that
> regard. It solves this particular problem and I can't see any
> downsides.
>
> Jeremy

The latest boot.rb in trunk does allow override by the env var.
However, this still only supports a specific version, not an advanced
version spec (e.g. floating on latest beta gem if env exists).

I'm not sure how easy it is to specify a advanced version spec (I've
never needed to), but assuming it can be specified as a string, it's
doable from the environment.

Also, as I said in the other thread, even if it the env var did take a
version spec instead of a specific version, it still wouldn't allow
cooler advanced hooks like auto-installation of missing gem versions
on app startup.

We are an agile shop that works on a lot of different rails projects,
so we switch pairs, machines and projects often. These are often on
different rails releases, which may or may not be installed on a given
machine. We also want to have our CI systems automate testing of all
our projects against multiple rails versions, including the latest
edge gems, without having to even think about it.

Given your scenario, my advice would be to wrap whatever command you
use to start Rails (mongrel_rails, style, etc.) in a script that
checks the gem version in the environment and downloads the gems
before starting rails if they are not currently installed.

Jeremy

#17

I'm not sure how easy it is to specify a advanced version spec (I've
never needed to), but assuming it can be specified as a string, it's
doable from the environment.

It's just the version prefixed with (in)equality or ~, see the link to
rubygems docs earlier in thread for more info.

The problem is that the current boot.rb hardcodes the prefix to "="
and assumes RAILS_GEM_VERSION. Now that we've discussed this, I
realize that it should really be more flexible. Given bitsweat's
excellent refactoring of boot.rb to make it testable and isolate
Rails::GemBoot.gem_version, we should go ahead and allow any rubygems
version spec in RAILS_GEM_VERSION. I think the default is equality,
so it may be as easy as removing the "=" (but should still be
well-tested anyway).

I'll go ahead and try to code up a separate patch to implement this
behavior. Even though it addresses some of my issues, it still
doesn't allow the flexibility that a preinit hook would. It will
prune my pitch for the preinit patch, but I'll be unselfish and
hopefully code and karma will prevail.

Given your scenario, my advice would be to wrap whatever command you
use to start Rails (mongrel_rails, style, etc.) in a script that
checks the gem version in the environment and downloads the gems
before starting rails if they are not currently installed.

That's a great idea, and what we currently do as part of our cap
scripts. As well as the beginning of our CI rake tasks. As for
mongrel, I suppose I could write a plugin, but that's more
duplication. Not to mention when we occasionally test under webrick
(handles PUTs differently), which doesn't have plugin support.

The point is, I want everything dependency-related (gem installation
and load path management) to be self-contained, be in one place, and
happen automatically whenever I run an app - whether that is during a
deploy, during local tests, during CI, or when I'm just firing up the
app via webrick or mongrel. Call me a pie-in-the-sky dreamer, but I
know it's possible. An _official_ pre-init hook is one of the key
requirements, though. Yes, I can continue to hack boot.rb, which
would probably be easier than repeatedly explaining the justification
for a preinit hook, but I started this thread and patch based on the
principle that I shouldn't be editing a file that tells me not to edit
it :slight_smile:

Thanks again for discussing, it's helping me to clarify things and
address the parts of the issue separately.

-- Chad

#18

This is done, with tests, in [10057].