Test coverage in Rails apps

rubyonrails-talk
1

How can I get test coverage results to show more consistent results that make sense?

I earlier decided against having SimpleCov and test coverage badges in my Rails apps. It seemed that no matter how I configured SimpleCov, there were some glaring flaws in its results. However, I have since changed my mind as a result of working on a legacy Rails app that had very sparse test coverage when I first joined. SimpleCov showed only 25% test coverage. As a result, I’ve decided to make test coverage a standard part of all Rails apps I’m on. Test coverage metrics may have their flaws, but I figure that test coverage results should flag the parts of the test suite that need the most improvement.

So I’ve added test coverage to my new Ruby.MN app. The production site is at http://rubymn2.herokuapp.com/, and the source code is at https://github.com/jhsu802701/rubymn2 . Currently, the CodeCov badge shows 93% test coverage, and the results make sense to me.

However, SimpleCov in my local development environment shows test coverage of only 76%. In fact, it doesn’t recognize ANY of my user or admin model tests. CodeCov does, and this is why it shows a much better test coverage figure.

What’s going on? Why is there such a big discrepancy? How can I configure SimpleCov to show results that make more sense?

2

In development run your tests with the DISABLE_SPRING=1 flag before the test command and see if you see a difference in coverage.

3

Thanks, Rob! Entering “DISABLE_SPRING=1 rails test” gives me the 93.16% test coverage.

Exactly what is the significance of the Spring server? Why isn’t Rails set up to automatically take care of this?

4

Spring is meant to make running locally (as in development and test) nearly as quick as running in production (where everything is cached). development isn't cached because it would mean you would have to stop the server and re-start every time you made any change at all. test is meant to be the same as production, but if they used the cache like production, the tests would take forever-long as each one would have to stop the process and start it up again (re-filling the cache) in order to be honest about one test not affecting any other. Spring is sort of a cache-lite, I guess, in that it allows reloading to happen, but caches some things that *should be okay to cache*. As you know, this is one of the Hard Problems in CS, so it's easy to imagine that some parts of the process do not agree with one another.

Walter