I have a test that fails 1 in 100 times and it’s a pain to wait for it to run so many times to reproduce the failure. I’d like to do lots of runs in parallel and stop when I hit the first failure. Does anyone know how to do this?
Why you have this randomness? It looks like a bad decision in your test. For me, a test has to be independent, well defined and repeatable.
Good unit test properties:
- It should be automated and repeatable.
- It should be easy to implement.
- Once it’s written, it should remain for future use.
- Anyone should be able to run it.
- It should run at the push of a button.
- It should run quickly.
It is not deliberate randomness! That is exactly what I’m trying to remove, but I have to be able to reproduce the failure reliably to do that. This is an End 2 End test of the UI through the browser all the way through the backend. I think the flakiness is being caused by visibility of an element in the browser, but that could be something in our code or a browser bug.
If is the same test, running it in parallel multiple would mean different databases and/or server instances, no?
Yes, ideally there would be 1 per core on the machine.