page cache more than 100K files inside a single directory?

That depends on your file system and has little to do with ruby or
rails. You're probably better off partitioning that somehow. Here's
what I usually do:

http://www.37signals.com/svn/archives2/id_partitioning.php