#8087 Vendor Everything

rubyonrails-core
1

Just trying to raise some awareness and discussion on a patch I recently posted.

The idea of freezing gems in your rails app has been around for a long time. However there is no suggested convention to follow. Chris Wanstrath has suggested the directory "vendor/gems" to be use in his post "Vendor Everything" (http://errtheblog.com/post/2120).

Once you set your path to this directory, everything works great. There is no code difference between loading gems from your local machine with rubygems or from the "vendor/gems" directory.

If everyone is down with this convention, we should add a few things in rails to encourage it.

1. Make the "vendor/gems" directory when you run the rails command. 2. Add "vendor/gems" to the load path 3. Load tasks from "vendor/gems" (I don't really use this but Nic Williams finds it useful) 4. Add some rake tasks to help you freeze gems.

Are there any other techniques out there that work well for you? Or does this sound like a delicious patch?

2

Link to the patch: http://dev.rubyonrails.org/ticket/8087

3

I loved Chris' post, I love this patch. +1. Here's to never deploying breaking production you forgot to install a gem or having to ssh to a bunch of machines to 'sudo gem install' (decidedly not DRY) ever again!

pt.

4

What about the native code issue? Are you going to create a sub-directory per platform?

Also, the idea of specifying dependencies (ala Maven) in yaml sounds pretty cool. Were you thinking about doing something like that?

Ben

Josh Peek wrote:

5

Gems already have a mechanism for specifying dependencies.

Several weeks ago when I checked in the changes to the plugin loading code Chad and I worked on functionality similar to the proposed patch. We took it a step further with a slightly different approach whereby gem files placed in a certain location would automaticaly be unpacked and loaded like regular plugins on startup. The code is complete (as we intended it), tested and documented, but despite Chad's repeated insistance, I haven't checked it in yet...I've attached a diff for those who are interested.

marcel

vendor_gems.diff (29.8 KB)

6

I meant the app's dependencies... which gems an app needs to have installed. Seems like there would need to be some outside help (new code and a config file) to do that.

b

Marcel Molina Jr. wrote:

7

The code we're working on will work for both plugins (as gems) and arbitrary gems.

Chad

8

Sweet!

b

Chad Fowler wrote:

9

Awesome! When will something like this make it into core?

10

Our main reservation vis-a-vis the vendor everything approach is that it makes for painful distribution when you have multiple teams and multiple applications that share the same 'operating environment' (i.e. colocated with a shared GEM_HOME).

If you have a major security hole in shared dependency Foo that you've backported to all of the major releases, you would need to hunt through every app and replace the gem in question with the right gem from based on release. Instead, we just build a gem for each backport, install them on the gem server, and the 'shared environment' need only update its repository and restart the apps. No need for a code deploy on the applications themselves, which would cause the need to create a maintenance branch on what might have been a short-lived release for the sole purpose of updating a dependency micro.

From a development perspective, repositories have worked out great for

us as well. When I fix a major bug in a shared dependency, my coworkers get the latest and greatest when they 'svn up; plugem up' (which leverages gem update) in the morning--no wack SVN externals (we started this journey in svn:external hell) or emails to links with the latest gem.

Platform-specific gems are also a dilemma in the vendor everything approach (rmagick, hpricot, etc.)--compiling things at application start seems somewhat unusual. We have both OSX developers and Windows developers, so it's nice to just add hpricot to the manifest and let the repository expose the different platform variations (as opposed to bundling them all in vendor and adding even more logic to the plugin stuff).

Gem repositories just do such a good job at acting as a repository for gems :slight_smile:

I know this isn't the world that everyone is in (or wants to be in for that matter), but I don't think it makes it much more difficult on the KISS crowd (while it opens up a lot of doors for us fools that like to overcomplicate things.)

The catch is the need for the application manifest file to declare the dependencies, but thats brings a lot of secondary value to the table as well. You can use it to generate spec files to gemify apps (as we do), guarantee the presence of required third-party gems at startup time, etc. It's not so bad.

More background on why we'd like to lobby for this approach in the body and comments of :

http://revolutiononrails.blogspot.com/2007/05/release-plugems-runtime.html http://revolutiononrails.blogspot.com/2007/05/plugems-more-than-dependency-management.html

Thoughts?

Eddie rails-trunk AT revolution.com

11

Update:

Val just released the packaging/update component... that + runtime should be all folks need to try plugems for themselves.

12

I patched edge with vendor_everything.diff but am getting this error:

uninitialized constant Gem::UnpackCommand /usr/local/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/rake.rb:2028:in `const_missing' /Users/shane/rails/contrib/vendor/rails/railties/lib/tasks/gems.rake:15

Did something happen to Gem::UnpackCommand?

Shane

13

What about the native code issue? Are you going to create a sub-directory per platform?

It's easy to compile C extensions per-platform. i.e.:

http://svn.kylemaxwell.com/rails_plugins/vendor_everything_extensions/tasks/vendor_everything_extensions_tasks.rake

This works in *nix-land, with a special case needed for Windows (as always).

Kyle