Have you decided what to use for indexing your documents? If not, you might like to consider SimpleSearch[1] and Ferret[2].
SimpleSearch works best for unstructured data whereas Ferret works best for structured data. Ferret is more sophisticated than SimpleSearch.
It sounds like the former suits your needs more unless the contents of your documents are structured.
Once you have decided how to index your documents, you need to actually do the indexing. I'd suggest doing that in a background process, so have a look at BackgrounDRb[3].
Hope that helps, Andy Stewart
[1] http://chadfowler.com/SimpleSearch [2] http://ferret.davebalmain.com [3] http://backgroundrb.rubyforge.org/
