Things I wish I knew when beginning testing

rubyonrails-talk
#1

The following statements will save you lots of time, and most
importantly, develop a better application when beginning Rails
development.

Fixtures are faster, but conciseness is more important. Use factories
(Machinist or Factory girl). Don't worry about the speed of your test/
spec suite. Use autotest or autospec.
--- I started out with fixtures, then switched to factories. Lost
patience and switched back to fixtures. Now back to factories for
maintainability and conciseness.

Do not mock everything to make your test/spec suite "database
independent". Your tests are SUPPOSED to break when you change code in
your model or controller. Mocking everything will not allow this, and
will give you the feeling that you're developing a second application
in your test/spec suite.
--- Yes, it will speed up your tests/specs, but once again,
conciseness is more important.

Your test framework does not matter. Test/unit, Shoulda, RSpec all
produce the same outcome. Choose by what syntax you feel comfortable
with. For every great developer that uses test/unit, there is a great
developer using RSpec.
--- For whatever reason I cannot explain, RSpec feels most comfortable
to me. I started with test/unit, switched to RSpec, then shoulda, then
test/unit, and now back to RSpec. I will not switch again.

Test the entire stack. Do not just test models. Test/spec models,
controllers, and use integration tests/specs. This will give you
confidence when developing and literally makes your work 1000 times
less stressful. Yes, it takes more time, but it is the ONLY way to
develop a reliable app that works.
--- I started out just testing models. This does not catch view
errors, and my users paid for it.

If TDD or BDD does not fit your brain, don't do it. DHH doesn't do it,
and you don't have to either. This is no excuse for not testing
everything though.
--- For me, TDD/BDD does not let me test/spec comfortably. I tried
both ways, and my code always turns out better when NOT using TDD/BDD.
I support the philosophy, but it is not "necessary". Some people will
say I am "doing it wrong", but if the end outcome is exactly what I
want, why does it matter how I got there?

If I had known these things when beginning, I would have saved
countless hours trying to follow the "one golden path" that does not
exist. I hope this helps.

#2

Thanks, but it’s too late :wink: Needed that advice a year ago. Until a few days back I had no idea how fun testing your app is. Love cucumber and the rest. It’s a good feeling when you know that you app is working well cause of all the tests you’re doing.

What’s also nice are the screencasts like peepcode.com, pragprog.com, bddcasts.com and teachmetocode.com etc. The community is just awesome.

Chris

#3

than you iam iranin ican not speek enghlis well &iwrith not hard fram halptime wery happiy maryam

#4

Test the entire stack. Do not just test models. Test/spec models,
controllers, and use integration tests/specs. This will give you
confidence when developing and literally makes your work 1000 times
less stressful. Yes, it takes more time, but it is the ONLY way to
develop a reliable app that works.
--- I started out just testing models. This does not catch view
errors, and my users paid for it.

Selenium is great for this, it even tests your javascripts. Things I
haven't to test with selenium is file-uploads and swf-movies. There
are hacks to test file-uploads, but I rather do controller tests for
those.

#5

What’s also important is refactoring (the third step in red/green/refactor). There a lot of real geeky tools like:

  • flog

  • reek

  • rcov

  • roodi

  • heckle

Cheers,

Chris

#6

Thanks for posting this -- it's some interesting food for thought.

Do not mock everything to make your test/spec suite "database
independent". Your tests are SUPPOSED to break when you change code in
your model or controller. Mocking everything will not allow this, and
will give you the feeling that you're developing a second application
in your test/spec suite.
--- Yes, it will speed up your tests/specs, but once again,
conciseness is more important.

I think this point is debatable though. The point of mocking models
in your tests is to allow the tests to run in complete isolation.
e.g., there's no reason for your Post model tests to break when you
add a new required field to your User model. Your point is valid for
integration and other high-level tests, but unit testing is equally
important in my mind and it certainly benefits from mocking.

Jeff

#7

Jeff,

Take the opposite of your scenario. Let's say you remove a column from
the User model......your tests still pass if its mocked. Do that a
couple times and you have cruft all over the application. That right
there gives me a uncomfortable feeling.....like I'm not really testing
my MY models.

#8

That's exactly my point! The unit tests for the Post model should be
blissfully ignorant of any and all changes to the User model. My
intent in unit testing a model is to simply verify that the behavior
stated actually occurs, *presuming that all the external factors
happen as they should.* If I have some code in the Post model that
sets a published flag only if the user is a moderator, I would rather
just say mock_model(User, :moderator => true) than go through the
process of building a moderator User model, handling validations,
required associations, etc. If a future change removed the moderator
flag from the User, then the tests I've written for the Post model
still say that the model works as it's written -- it's just that the
User no longer conforms the the Post's expectations. Thus, I expect
damaging changes to the User model not to show up in my Post unit
tests, but in some various high-level integration tests where I'm
actually observing that the interactions between models go as planned.

I agree that mocking can leave your tests a bit crufty unless you're
very skilled at mocking up front (which I definitely am not :)).
However, in my experience, when I start working with unit tests for
complex models without mocks, I find myself spending too much time
just maintaining the tests for the changes occurring in the associated
models. I find the overhead of handling mocks preferable to keeping
up with changes in other models.

Mocking is something I've spent an inordinate amount of time
considering, and I have come to terms with the pros (complete
isolation) and cons (test management overhead). I've also come to the
conclusion that this is very much like the TDD/BDD/EDD/whatever flavor
is popular argument -- there's some substance to the differences, but
at the end of the day it's a matter of personal taste.

Cheers!
Jeff