2.1 Gem Dependencies

I seem to be doing something wrong w/ gem dependencies: if I put a
statement in environment, and then write some code to use that gem in a
rake task or in a controller (such as a 'require' statment), then rake
will no longer work. It is broken, because the gem is not installed.
But, I can't install the gem, because rake is broken when it tries
to load the envioronment.

How do I get around this? This seems like a core deployment scenario, I
can't help but feel like I am missing some key part of 2.1 gem

Here's a specific example:

run: rails /home/user/test/
edit: enviornment.rb, uncomment (or add) the line: config.gem "bj"
add: test.rake to ./lib/tasks
edit: test.rake:

namespace :test do
  require 'bj'
  desc 'test'
  task :test => :environment do
    puts Time.now

run: rake gems:install

the last command will fail, because the bj gem is not there.