There are up to two per day for each user, but they can be handled
separately. Yes, currently using the disgusting "find(:all)" approach.
Since you mentioned using a background task for this, I assumed a
straight SQL query. My bad.
No, with just 100 users so far there is no sign of a problem. I just
consider the current method to be very inelegant and would rather do it
in a more intelligent way. Even if, once we get to a few thousand
users, it still isn't causing a "problem", it will be using significant
resources, thus reducing the responsiveness of the app.
My point, which I attempted -- apparently unsuccessfully! -- to make
humorously, is that running a single SQL qeuery against a properly
indexed database, with even a row count in the millions, is trivial.
Really. And if it's being done from a background app, should have
little or no effect on your Rails app's performance from the user's
So is it generally the case that having the db hunt down the few records
we're after each time will execute substantially faster than returning
all of the records and doing the comparison "outside the database"?
Absolutely -- that's what an RDBMS is designed for.
I would think that some method of caching the info such that it didn't
have to actually go to the db every time would make the biggest
difference in execution time. Unfortunately, such a solution would also
be more complex than just using a better query.
The execution time to retrieve the list and build the queue is trivial
compared to the amount of time to actually process the mail queue,
particularly as the user count grows.
Still, for the sake of argument -- build your "mailing list" in memory,
and update it when a change event occurs in the app (user submits
a changed profile, new user joins, etc.). Though you'll still incur at
least one update/insert into the DB, which is more expensive than a
One last venom-free comment regarding premature optimization --
if you "fix" something today that /isn't/ broken, you may be making it
harder to fix a /real/ bottleneck that crops up tomorrow.
Just sayin' ...
Best of luck,