#8087 Vendor Everything

Just trying to raise some awareness and discussion on a patch I
recently posted.

The idea of freezing gems in your rails app has been around for a long
time. However there is no suggested convention to follow. Chris
Wanstrath has suggested the directory "vendor/gems" to be use in his
post "Vendor Everything" (http://errtheblog.com/post/2120).

Once you set your path to this directory, everything works great.
There is no code difference between loading gems from your local
machine with rubygems or from the "vendor/gems" directory.

If everyone is down with this convention, we should add a few things
in rails to encourage it.

1. Make the "vendor/gems" directory when you run the rails command.
2. Add "vendor/gems" to the load path
3. Load tasks from "vendor/gems" (I don't really use this but Nic
Williams finds it useful)
4. Add some rake tasks to help you freeze gems.

Are there any other techniques out there that work well for you? Or
does this sound like a delicious patch?

Link to the patch: http://dev.rubyonrails.org/ticket/8087

I loved Chris' post, I love this patch. +1. Here's to never
deploying breaking production you forgot to install a gem or having
to ssh to a bunch of machines to 'sudo gem install' (decidedly not
DRY) ever again!

pt.

What about the native code issue? Are you going to create a
sub-directory per platform?

Also, the idea of specifying dependencies (ala Maven) in yaml sounds
pretty cool. Were you thinking about doing something like that?

Ben

Josh Peek wrote:

Gems already have a mechanism for specifying dependencies.

Several weeks ago when I checked in the changes to the plugin loading code
Chad and I worked on functionality similar to the proposed patch. We took it
a step further with a slightly different approach whereby gem files placed
in a certain location would automaticaly be unpacked and loaded like regular
plugins on startup. The code is complete (as we intended it), tested and
documented, but despite Chad's repeated insistance, I haven't checked it in
yet...I've attached a diff for those who are interested.

marcel

vendor_gems.diff (29.8 KB)

I meant the app's dependencies... which gems an app needs to have installed. Seems like there would need to be some outside help (new code and a config file) to do that.

b

Marcel Molina Jr. wrote:

The code we're working on will work for both plugins (as gems) and
arbitrary gems.

Chad

Sweet!

b

Chad Fowler wrote:

Awesome! When will something like this make it into core?

Our main reservation vis-a-vis the vendor everything approach is that
it makes for painful distribution when you have multiple teams and
multiple applications that share the same 'operating
environment' (i.e. colocated with a shared GEM_HOME).

If you have a major security hole in shared dependency Foo that you've
backported to all of the major releases, you would need to hunt
through every app and replace the gem in question with the right gem
from based on release. Instead, we just build a gem for each backport,
install them on the gem server, and the 'shared environment' need only
update its repository and restart the apps. No need for a code deploy
on the applications themselves, which would cause the need to create a
maintenance branch on what might have been a short-lived release for
the sole purpose of updating a dependency micro.

From a development perspective, repositories have worked out great for

us as well. When I fix a major bug in a shared dependency, my
coworkers get the latest and greatest when they 'svn up; plugem
up' (which leverages gem update) in the morning--no wack SVN externals
(we started this journey in svn:external hell) or emails to links with
the latest gem.

Platform-specific gems are also a dilemma in the vendor everything
approach (rmagick, hpricot, etc.)--compiling things at application
start seems somewhat unusual. We have both OSX developers and Windows
developers, so it's nice to just add hpricot to the manifest and let
the repository expose the different platform variations (as opposed to
bundling them all in vendor and adding even more logic to the plugin
stuff).

Gem repositories just do such a good job at acting as a repository for
gems :slight_smile:

I know this isn't the world that everyone is in (or wants to be in for
that matter), but I don't think it makes it much more difficult on the
KISS crowd (while it opens up a lot of doors for us fools that like to
overcomplicate things.)

The catch is the need for the application manifest file to declare the
dependencies, but thats brings a lot of secondary value to the table
as well. You can use it to generate spec files to gemify apps (as we
do), guarantee the presence of required third-party gems at startup
time, etc. It's not so bad.

More background on why we'd like to lobby for this approach in the
body and comments of :

http://revolutiononrails.blogspot.com/2007/05/release-plugems-runtime.html
http://revolutiononrails.blogspot.com/2007/05/plugems-more-than-dependency-management.html
http://interblah.net/2007/5/2/plugems-enter-the-rails-addon-fray/comment/297#comment-297

Thoughts?

Eddie
rails-trunk AT revolution.com

Update:

Val just released the packaging/update component... that + runtime
should be all folks need to try plugems for themselves.

http://revolutiononrails.blogspot.com/2007/05/release-plugems-packaging.html

I patched edge with vendor_everything.diff but am getting this error:

uninitialized constant Gem::UnpackCommand
/usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:2028:in `const_missing'
/Users/shane/rails/contrib/vendor/rails/railties/lib/tasks/gems.rake:15

Did something happen to Gem::UnpackCommand?

Shane

What about the native code issue? Are you going to create a
sub-directory per platform?

It's easy to compile C extensions per-platform. i.e.:

http://svn.kylemaxwell.com/rails_plugins/vendor_everything_extensions/tasks/vendor_everything_extensions_tasks.rake

This works in *nix-land, with a special case needed for Windows (as always).

Kyle