The Rails test engine contains otherwise flawless error handling. So, after being bedeviled by intermittent "Test failure" abnormal test terminations, I finally got around to --trace-ing the problem, and when I found it the clumsiness surprised me:
desc 'Test all units and functionals'
task :test do
Rake::Task["test:units"].invoke rescue got_error = true
Rake::Task["test:functionals"].invoke rescue got_error = true
Rake::Task["test:integration"].invoke rescue got_error = true
raise "Test failures" if got_error
What that means is, if any test_ case fails, we see a healthy error message, but if the test rig itself fails, 'rescue got_error = true' will throw away the e.message and e.backtrace, and the raise "Test failures" will provide an irrelevant error message and backtrace.
I'm going to copy this task :test into my lib/tasks/project.rake file, and replace the error hiding with a real error handling block. Further, if the test rig itself has failed, I don't care to keep running any more tests, so I won't bother with the system that tolerates their faults.
Why was this method so clumsy, and hasn't anyone asked for an upgrade on it yet?