Anthony Eden wrote:
I don't see how a test can be written to show something won't be
fixed. I also don't how a test can prove an issue doesn't exist.
A passing test case which demonstrates that specific input results in
specific output indicates that an issue doesn't exist, at least using
the assumptions of the original issue. Granted a test case may pass in
one case and fail in another due to variations in the environment
(OSes, configurations, etc) however a test is better than no test. As
for test cases for something which won't be fixed, at least point to
or provide a test case which demonstrates the expected behavior. If
there is no test case then the correct behavior is a matter of
interpretation and will often lead to confusion, bugs, more people
raising issues, etc.
If tickets are closed with 'works for me', the closer couldn't think
of a test that fails. It doesn't prove the issue doesn't exist, but
then, proving a negative isn't exactly easy. The onus should be on
those with an interest in reopening the ticket to provide a failing
test, rather than on those closing tickets to write passing tests.
How do you know that the closer couldn't think of a test that fails?
Unless the closer creates a test that passes based on a set of
assumptions then how do you know what lead them to believe that it
works for them? We don't know what the closer's assumptions were, and
the point of test cases is to define something which can be applied
repeatedly with the same assumptions and which results in the same
outcome.
I think if an issue can be legitimately closed with a passing test
case then the onus does reasonably shift to the original person who
wrote the ticket to provide a test case that fails.
You are right to point out that variations in environment can cause a test to fail for the creator of a ticket, but succeed when tested by a Rails committer.
But the situation you are concerned about seems to be when the ticket creator hasn't given a failing (for him/her) test. You are proposing that the Rails committer who wants to close a ticket (i) interprets the reported problem in terms of a test, (ii) writes the test, and (iii) demonstrates that, in some 'standard' environment, the test passes.
In other words, you are proposing that Rails is presumed guilty until proved innocent, and asking the committers to act as the lawyers for the defendant.
The number of Rails users is in the tens or hundreds of thousands, with a wide variety of levels of skill and experience. You can see from reading the mailing list how often people raise 'problems' that turn out to be the result of their own error or misunderstanding.
The number of committers is small, and they are highly skilled. In the circumstances, the committers *must* be able to close tickets without spending a lot of time explaining why, and the burden must then fall on the ticket creator to justify reopening the ticket. Anything else doesn't make economic sense.
regards
Justin Forder
