hi, when do you start worrying about scaling and caching?

Monitoring is the first step. I use the Robot Co-Op's libraries for analyzing Rails log files when written in a special format:

http://nubyonrails.com/articles/2007/01/03/a-hodel-3000-compliant-logger-for-the-rest-of-us