What do you do? A new release can break an application build on a preceding release and often an application needs to modify. So what to do? Fix the rails version in the Gemfile? In this manner I'll have a lot of rails versions in the disk.
I keep things fixed and upgrade when I have had the time to check things out. A few extra MB of gems on disk isn't really relevant to me.
Fred
I create a new gemset, a new branch and upgrade. Fix what needs to be fixed. When everything is OK, merge the branch, delete the unused gemset. (If you want to save some disk space. Could be the old one or the new one, that's up to you)
Hope it helps.