If you add an exit to a controller, in a place that will be executed when the file is loaded and then try and run tests, the test runner still outputs the line of how many tests were run/errored/failed, but it just says zero for all of them, regardless of the number of tests, which is confusing and difficult to diagnose.
1 Like
I love this! This is exactly the kind of WTF that a) I as a Ruby expert would never even think to try (exit in a controller); but b) is completely reasonable for a newbie to try.
Initial brainstorming: I wonder if we could add a SystemExit handler within the request/response loop that outputs a warning?
I’m not sure that would help with the test case… but for tests, we could detect if exit occurred for some reason other than the runner finishing.
Right? I was trying to help a student with their tests and got stumped by this for a very long time and wasn’t even the person who finally noticed the exit. 
I just made a blank rails app here to show the problem: GitHub - dougcole/rails-may-wtf: example for https://discuss.rubyonrails.org/t/adding-an-exit-in-a-controller-and-then-running-tests-shows-0-tests-run-but-no-errors/74363 . I actually have a little time and could help with the fix (I’m unemployed 
) , but I’m not familiar with the test runner internals, so thought I’d post it here for starts in case someone who knew the code better had an easy fix!
Thanks for the example repo! That’s super helpful.
I think the issue here isn’t actually in the request/response loop, but during the loading of the controller. If an exit happened there it wouldn’t be perfect but actually exiting would at least be less confusing .
Why this happens is totally confusing 
! The way tests are run under minitest (or specifically when minitest/autorun is being used) is during an at_exit handler. Inside this handler is where all tests are normally run.
I had to check in a console. It seems exit raises a SystemExit, but “running out of code to run” doesn’t raise anything.
$ ruby -e 'at_exit { p $! }'
nil
$ ruby -e 'at_exit { p $! }; exit'
#<SystemExit: exit>
The at_exit handler seems to explicitly handle running tests which are a SystemExit (ie. from an exit):
next if $! and not ($!.kind_of? SystemExit and $!.success?)
I wonder if Minitest.autorun needs to be running on an exit/SystemExit, or if that line could just be next if $! and only run when “all code is done loading”. Of course there might be a good reason it can’t 
.
3 Likes
This kind of Ruby exception/exit minutiae is sort of my bag, so let me know if you have any particular questions.
