Add `assert_raises_with_message` testing helper

error = assert_raises(SomeError) do
  do_something
end
assert_equal "Some error message", error.message

is a very popular pattern in tests. I found 373 matches of it in the Rails itself. There are even 2 implementations of it in the codebase: rails/finder_test.rb at 12ae2e2e648dc4c26da777e7809504a8d2cde962 · rails/rails · GitHub and rails/json_gem_encoding_test.rb at 12ae2e2e648dc4c26da777e7809504a8d2cde962 · rails/rails · GitHub

There was once a suggestion in minitest for this - Interest in assert_raises_with_message? · Issue #530 · minitest/minitest · GitHub, but it was decided (6 years later) that is not very much needed. But I still think this can simplify code a bit.

This change will also help to avoid a common pitfall when people pass the last argument (as a string) to assert_raises thinking that it will be matched against the error too. Personally, I was in this trap several times.

I even have such helper in my own gem - online_migrations/minitest_helpers.rb at e8dab9f6dea1e670a3fb086a03c211c38c50aeec · fatkodima/online_migrations · GitHub.

3 Likes

I would say that asserting a message along with an exception class should be the only way to assert an exception. Otherwise, the test is under risk of passing while asserting for the same error, but raised in a completely different part of the code. For example - Fix postgresql reconnection_error test expects wrong exception by nvasilevski · Pull Request #44347 · rails/rails · GitHub

So definitely +1

1 Like

Agreed!

Not related to my original suggestion, but I also love when people test exceptions like this:

assert_raises(ArgumentError) do # or RuntimeError or StandardError etc
  # ...
end

with no message testing. I have found many examples of this in the Rails codebase. You will just incorrectly call a method and get a false positive.

Or another example:

assert_raises(SomeError) do
 # many-many lines of ruby code
end

In this case it is very possible to get an error not from where you are expecting.

Fortunately, there is a minitest rubocop cop for one of them - Add new `Minitest/AssertRaisesCompoundBody` cop by fatkodima · Pull Request #177 · rubocop/rubocop-minitest · GitHub and am going to implement another one - New cop to check `assert_raises` with generic exceptions is followed by error message check · Issue #174 · rubocop/rubocop-minitest · GitHub.

1 Like

I’d be in favor of

assert_raises MyError, match: /my message/ do

end

with match: basically calling assert_match match, error.message.

The reason I think match is important is that with error_highlight and such, it’s not common to have unpredictable stuff appended to the message, so you don’t want to do an exact comparison most of the time.

Yes, looks more flexible. match looks a little vague to me, because we are already trying to kinda 'match’ing exceptions, it is not probably exactly clear what this match does different. But I do not have a better name.

So, do you suggest to try to push this forward to the minitest library itself?

match looks a little vague to me, because we are already trying to kinda 'match’ing exceptions

The alternative is message: /foo/, but I fear it may be confusing with the assertion message.

So, do you suggest to try to push this forward to the minitest library itself?

You can try if you wish, but the maintainer is rarely open to new feature. Best is probably to submit the feature in both Active Support and Minitest. We can add this to Active Support and remove it later if it’s accepted upstream.

1 Like