Liz, the point above about using RVM (or similar) is that you can install in your user account the latest versions of ruby and rails without issue, however, being a longish user of Dreamhost myself (although I dont host ‘customer’ websites there myself), because of their lack of official support for later stuff (last time I looked)
That being said however, you can get a VPS with someone like Digital Ocean (or many others) for anywhere from $5 a month upwards, and for a small traffic site, that is probably more than enough.
If you are not doing ‘devops’ functions (i.e you know how to code in Ruby/Rails but are not aware of all the nuances of running a server - patching/upgrades etc), then you may need to learn a few things, however, if you want to talk to me off-list I can give you some pointers. Looking at Dreamhosts current offfers, you are probably paying more than $5 a month for the hosting anyway.
Depending on where you want to host (country), you can find lots of very good offers on www.lowendbox.com, which will give you VPS you (virtually) control, so you can install what you need on them for (in most cases) %5-$10 per month.