why detest rspec? (was: community poll about testing and specs)

At the risk of shooting myself in the foot, I'm really curious to know
why you detest RSpec.

I didn't like it for a while because the syntax kept changing. This
made me rewrite the BDD section of my book twice (three times when I
rewrite to match 1.0).

But now that I've been using the new syntax more, I like it. :stuck_out_tongue:

--Jeremy

@Roderick:
Wow. That mirrors my thoughts exactly. I’d love for someone to chime in and counter this. Thanks for putting it so well.

Roderick: David Chelimsky might be a little miffed with your selective quoting which makes it appear as if he doesn’t like RSpec…

David Chelimsky wrote:

I may be biased, but I would really like and have been looking for an

alternative BDD choice besides RSpec.

This was not written by David, but was written in response to an email from him. The only clue to this is the level of indentation of the quoting.

Alsak demonstrated it to be a wrapper DSL around Test::Unit
for "people who find it hard to come to gripes with TDD". So it's a sort
of workaround? I was disappointed by that statement of his.

I don't get it. What can you write in RSpec that you can't write,
nearly the same way, in Test::Unit?

He says Test::Unit is tightly coupled to the actual code and that that's
a bad thing.

Done right, either one tightly _coheres_ to the actual code.

(Define "couple" as "A must change only because B changed", and define
"cohere" as "A and B share legitimate reasons to change together".)

David Chelimsky wrote:
>> I may be biased, but I would really like and have been looking for an
>> alternative BDD choice besides RSpec.
>>
>> What others are there? I have seen a couple by the likes of Ryan, even
>> Rick. Are any better? What about a conrast of testing libraries
>> highlighting the benefits and opinions of each?
>>
>> I really want to get into testing beyond Test/Unit, but I detest the
>> RSpec library out of spite :slight_smile:
>
> At the risk of shooting myself in the foot, I'm really curious to know
> why you detest RSpec.

I'd like to chime in. I would not at all say that I detest RSpec in any
way, but I do find it to be rather besides the point. It just never
caught on to me, you know?

When I first looked at RSpec, I thought it was rather cryptic. This
boils down to a matter of taste, I know, but all those "recipes" (I
don't know the term) really aren't all that legible to me.
"bowling.score.should == 0"? "should" is equal to 0? Is it any more
expressive than "assert 0, bowling.score"? Again, it's personal
preference, but this is one DSL that I don't care for. So I dismissed it
in favor of the Test::Units I have come to cherish.

Then I attended a lecture by Aslak Hellesøy. I mean, with so many people
around raving about RSpec, I must be missing the point right? But I was
let down: Alsak demonstrated it to be a wrapper DSL around Test::Unit
for "people who find it hard to come to gripes with TDD". So it's a sort
of workaround? I was disappointed by that statement of his.

He says Test::Unit is tightly coupled to the actual code and that that's
a bad thing. I say I like that coupling, because it keeps a sharp edge
on the test-before-you-write principle: if you're mucking around code or
tests that worked before, you'll find yourself pressed to advocate that
change and express it in an additional or revised test.

Moving from personal preference into the field of computer science,
there is no argument that I know of favoring either in code coverage or
testing metrics. (Indeed, "coupling" and "cohesion" have so far only
referred to the functional components themselves, not their tests.) This
doesn't prove either Alsak or me wrong or right, but does iterate that
it's a matter of preference.

"But," some might say, "we're testing *behavior* here. That's different.
We can clean up methods, throw code around, but as long as the behavior
remains the same, then that's OK." I see it as the same discussion about
white box testing and black box testing, but with a twist.

As a black box, the bowling example on the RSpec home page seems to work
fine. As a white box, it's lacking in every respect because the
algorithm we assume to be present is, in fact, completely absent. I like
to test the algorithm, because if I don't prove the behavior of the
algorithm to be correct then I will need to verify every possible
outcome instead. That kind of verification isn't very efficient.

So the twist is this: this time around, we're mixing up black box
testing by dubbing it "behavior" and white box testing by performing
actual method calls. I've been searching for a rationale, but could only
find the one by Alsak that I was disappointed by before.

The way I see it, RSpec is just a matter of preference, and a bit of a
hype at that. Perhaps I'm misunderstanding, and if so, I'd love to see
the entire mantra clarified. But until then, I'm preferring Test::Unit
for its readability and code coupling, and saying nay to all those who
preach that RSpec is a level beyond Test::Unit.

I'm sorry that you see this as preaching and mantras. I have no
intention of converting you. I like RSpec because it expresses the
things I want in a BDD framework. You don't have to like it.

BDD comes from TDD, which comes from XP, which includes Refactoring as
an integral part of the process. Refactoring is more expensive as your
tests are more tightly coupled to the code, so one goal of TDD is to
decouple the tests from the detail of the code (which is what changes
most often). One thing that aids in this decoupling is focusing on
behaviour instead of internal implementation. BDD champions this goal
by putting it front and center. RSpec supports this by trying to use
words like "describe" instead of "TestCase" and "should" instead of
"assert."

It is clear from your arguments that a) your testing goals do not
align with those of RSpec and b) you've only read the cover (your
argument about the example on the Home Page). If you're seriously
interested in learning about RSpec and BDD, then you should try to
actually use it on a small project and see how it feels. If not, then
so be it. I still get to use it, so I am happy.

Cheers,
David

Roderick: David Chelimsky might be a little miffed with your selective
quoting which makes it appear as if he doesn't like RSpec...

David Chelimsky wrote:
>> I may be biased, but I would really like and have been looking for an
>> alternative BDD choice besides RSpec.

This was not written by David, but was written in response to an email from
him. The only clue to this is the level of indentation of the quoting.

Actually, this makes me laugh more than piss me off. But thanks for
sticking up for me :slight_smile:

Cheers,
David

@David:

Would you be kind enough to briefly explain what you perceive to be the advantages to RSpec? I’m really hoping for a good comparison of the two, so I can see if it’s worth investigating further. (I am writing a book that will include testing, and need more info). I’ve played with RSPec and haven’t really found anything I can’t do with test:unit… so that’s why I’m looking for more info. I’d really appreciate some input. I suspect that it’s much like TDD… you have to do it before you really appreciate it.

I don't know that there is anything that you can't do with test/unit -
just that rspec comes w/ a lot of this stuff out of the box. And I
think it is something experiential. Many people who really enjoy using
RSpec say they were skeptical at first, but have grown to love it.

Here are some specific comparisons:

- dsl for expressing/organizing examples using strings

describe Thing do
  it "should do cool stuff" do
    ...
  end
end

vs

class ThingTest << Test::Unit::TestCase
  def test_should_do_cool_stuff
    ...
  end
end

I find that this encourages better naming and organization.

- dsl for expressing expectations

team.should have(11).players

vs

assert_equal 11, team.players.size

- better failure messages

expected empty? to return true, got false

vs

expected true, got false

- better reporting

Thing
- should do this
- should do that
- should do the other thing (FAILED - 1)
- should do even this (PENDING: Not Yet Implemented)

vs

...............

There is also a nicely formatted HTML report out of the box:

  http://rspec.rubyforge.org/report.html

- rspec_on_rails - component isolation allows you to test views before
controllers or models even exist. This is VERY helpful when you're on
an XP or XP-like team.

I'm sure there's more that I'm not thinking of right now. There's also
the "joy" factor. I find it a more joyful experience to use RSpec, for
the reasons stated above and for reasons that you can only appreciate
by exploring it.

Hope this helps.

Cheers,
David

As an aside, RSpec's use of "should" may be confusing to anyone coming from the networking world:

http://www.faqs.org/rfcs/rfc2119.html

-faisal

> RSpec supports this by trying to use words like "describe" instead
> of "TestCase" and "should" instead of "assert."

As an aside, RSpec's use of "should" may be confusing to anyone
coming from the networking world:

http://www.faqs.org/rfcs/rfc2119.html

:frowning:

Can't clarify everything for everybody I guess.

Brian Hogan wrote:

@David:

Would you be kind enough to briefly explain what you perceive to be the
advantages to RSpec? I'm really hoping for a good comparison of the two, so
I can see if it's worth investigating further. (I am writing a book that
will include testing, and need more info). I've played with RSPec and
haven't really found anything I can't do with test:unit....

And that's as expected. As you'll read in most intros to BDD, it's the same practice as TDD, but with the focus shifted. Everything you can do with BDD can be done with TDD.

People seem to be expecting some sort of sea change in development practices, but that's not going to happen here. And from what I've read, it was never intended to.

It's a small tweak, to refocus TDD practitioners a slight bit. They think it produces better specs/tests, not because it's something that couldn't be done before, but because it's now more obvious what you should be doing.

If TDD has always worked well for you (as I've read before, "if you've
been doing it right all along"), then great, stick with it.

But the very simple terminology changes are what made the point of TDD finally click for me.

As for rspec specifically? It does what I need it to, produces phenomenally legible specs, and then stays out of the way.

Kyle

@David, @Kyle:

Great stuff. If you were going to expose people to tests, would you start with TDD and test:unit, or would you go right to BDD and RSpec?

Green field? i.e. no experience with testing at all? Definitely RSpec.

Beyond that, it depends on myriad factors, mostly boiling down to what
is going to be most pragmatic for the team based on their level of
experience with Ruby, testing in general, other frameworks, etc. I'd
always prefer RSpec because, well, I prefer it. But sometimes going w/
test/unit could be more pragmatic, at least in the short run.

David

This argument is the same one you could use to say “what can you write in Ruby that you can’t write, nearly the same way, in C”? Heck, they are both TC general purpose computer programming languages. Ahhhh, but the “feel” is different. That’s what I find sets RSpec apart, and if it doesn’t click for you it might not work for you.

s.ross wrote:

This argument is the same one you could use to say "what can you write in
Ruby that you can't write, nearly the same way, in C"?

Uh, in the same volume of code? At the same velocity?

OK, so what about python?

I think the point Steve was making was about "feel", which is what
Ruby and Rails are largely about. To me, there is a huge difference
between this:

team.should have(11).players

  and this

assert_equal 11, team.players.size

And an even bigger difference in the output when these fail:

expected 11 players, got 10

  vs

expected 11, got 10

Implementing these probably takes the same amount of time. A month
later, seeing "expected 11 players" is VERY meaningful, especially
when multiple failures appear due to a change you just made. If I see
10 failures that all say "expected x things, got y", then I have a lot
more information right away about what went wrong than if I just see
"expected x, got y".

FWIW.

David

I'd like to chime in. I would not at all say that I detest RSpec in any
way, but I do find it to be rather besides the point. It just never
caught on to me, you know?

When I first looked at RSpec, I thought it was rather cryptic. This
boils down to a matter of taste, I know, but all those "recipes" (I
don't know the term) really aren't all that legible to me.
"bowling.score.should == 0"? "should" is equal to 0? Is it any more
expressive than "assert 0, bowling.score"? Again, it's personal
preference, but this is one DSL that I don't care for. So I dismissed it
in favor of the Test::Units I have come to cherish.

Cryptic? Seriously?
"bowling score should equal 0"
vs
"assert 0 bowling score"

The first statement is parseable by a second grader. The second
statement only makes sense if you are very familiar with test::unit.

I think the problem is that you're looking at the code and ignoring
the meaning. You ask "'should' is equal to 0?" It seems that you're
examining the code the way the interpretter would, and thinking, "the
object returned by the method should is equal to 0." That's
completely the wrong mindset.

I would say that TDD is a tool to help you solve the problem of
designing and implementing behavior. Test::Unit works fine in that
regard, but RSpec reduces the semantic distance between the developer
and the problem domain.

Consider a task list you write for yourself. Again you'll have
statements that look like "bowling score should equal 0" rather than
"assert 0 bowling score." Throw in a couple dots and you're good to
go :slight_smile: RSpec lets you express behavior in code almost as neatly as you
do in your head. That's very, very powerful. I've yet to come across
a tool that brings you that close.

Then I attended a lecture by Aslak Hellesøy. I mean, with so many people
around raving about RSpec, I must be missing the point right? But I was
let down: Alsak demonstrated it to be a wrapper DSL around Test::Unit
for "people who find it hard to come to gripes with TDD". So it's a sort
of workaround? I was disappointed by that statement of his.

I'm not sure how long ago that lecture was, but to the best of my
knowledge RSpec is no longer just a wrapper around Test::Unit. On the
other hand, I'm not sure why it matters how it's implemented under the
hood. We are concerned with behavior here, after all :wink:

Anyway, the big BDD guys all basically say that there's nothing
different if you're doing TDD right. BDD/RSpec isn't a workaround to
TDD, but rather a fast track to the good things about TDD. Again you
have the benefit of a shorter semantic distance. By using RSpec you
start off focusing on design and behavior, which is the ultimate goal
of TDD.

He says Test::Unit is tightly coupled to the actual code and that that's
a bad thing. I say I like that coupling, because it keeps a sharp edge
on the test-before-you-write principle: if you're mucking around code or
tests that worked before, you'll find yourself pressed to advocate that
change and express it in an additional or revised test.

The concern is that Test::Unit encourages an artificial coupling of
one test class to one production class. That's a completely different
thing from tests being coupled to the code, which as you pointed out
will/should be the case. The problem is that a new test class can
feel heavy weight (especially if you've been using JUnit for several
years, you almost certainly have that stigma). People think, "wow why
should I create a whole new class when it's just a little bit of
behavior?" Most people know that it's not a good idea to have a class
with just one or two methods, but testing has different rules and the
fact is that a test class with just a few methods is often precisely
what you need.

RSpec helps you avoid this problem by encouraging you to write several
small, focused examples. Once again it's not something that you
couldn't do with Test::Unit, but the RSpec way is just a lot closer to
the Right Way.

Moving from personal preference into the field of computer science,
there is no argument that I know of favoring either in code coverage or
testing metrics. (Indeed, "coupling" and "cohesion" have so far only
referred to the functional components themselves, not their tests.) This
doesn't prove either Alsak or me wrong or right, but does iterate that
it's a matter of preference.

"But," some might say, "we're testing *behavior* here. That's different.
We can clean up methods, throw code around, but as long as the behavior
remains the same, then that's OK." I see it as the same discussion about
white box testing and black box testing, but with a twist.

As a black box, the bowling example on the RSpec home page seems to work
fine. As a white box, it's lacking in every respect because the
algorithm we assume to be present is, in fact, completely absent. I like
to test the algorithm, because if I don't prove the behavior of the
algorithm to be correct then I will need to verify every possible
outcome instead. That kind of verification isn't very efficient.

I don't think RSpec is meant to be a white-box testing tool. Again,
it's all about design and behavior. Despite that, you should still be
able to achieve a sufficient level of white-box testing using mocks.

In fact, sometimes I'll write two sets of specs for the same example.
One will use concrete implementations, allowing me to verify that I
get the desired side-effects (records created, etc). Another one will
use mocks, allowing me to verify that the proper interactions are
being made. I've got a good example, but I don't want to add 65 lines
of code to this already lengthy message...but if there's any interest
I'll be happy to post it in a followup message.

So the twist is this: this time around, we're mixing up black box
testing by dubbing it "behavior" and white box testing by performing
actual method calls. I've been searching for a rationale, but could only
find the one by Alsak that I was disappointed by before.

The way I see it, RSpec is just a matter of preference, and a bit of a
hype at that. Perhaps I'm misunderstanding, and if so, I'd love to see
the entire mantra clarified. But until then, I'm preferring Test::Unit
for its readability and code coupling, and saying nay to all those who
preach that RSpec is a level beyond Test::Unit.

Sure, just as some people prefer C. It's certainly more readable -
you have semicolons that tell you where a statement ends!

Pat

We could debate either side of the issue. I could say “with the same compactness? with equivalent performance characteristics?” but at the end of the discussion, it’s all about what clicks for you. If you are happy with asserts (like in C, hence my analogy), then use them. I believe the BDD people – and I count David among one of the most thoughtful of the bunch – have put a good deal of work into making the description of the spec as natural as possible. Again, it’s what reads naturally to you. If you don’t parse it naturally, then no prob. Also, make careful note of what David says about the failure messages. This was one of my biggest complaints with Test::Unit and the fix is pretty close to free with RSpec. The failures report pretty much in English and if they don’t I probably wrote the example wrong.

–s

Pat Maddox wrote:

In fact, sometimes I'll write two sets of specs for the same example.
One will use concrete implementations, allowing me to verify that I
get the desired side-effects (records created, etc). Another one will
use mocks, allowing me to verify that the proper interactions are
being made. I've got a good example, but I don't want to add 65 lines
of code to this already lengthy message...but if there's any interest
I'll be happy to post it in a followup message.

This sounds very interesting. I would love to see a concrete example detailing your approach.