At the risk of shooting myself in the foot, I'm really curious to know why you detest RSpec.
I didn't like it for a while because the syntax kept changing. This made me rewrite the BDD section of my book twice (three times when I rewrite to match 1.0).
But now that I've been using the new syntax more, I like it. 
--Jeremy
@Roderick: Wow. That mirrors my thoughts exactly. I’d love for someone to chime in and counter this. Thanks for putting it so well.
Roderick: David Chelimsky might be a little miffed with your selective quoting which makes it appear as if he doesn’t like RSpec…
David Chelimsky wrote:
I may be biased, but I would really like and have been looking for an
alternative BDD choice besides RSpec.
This was not written by David, but was written in response to an email from him. The only clue to this is the level of indentation of the quoting.
Alsak demonstrated it to be a wrapper DSL around Test::Unit for "people who find it hard to come to gripes with TDD". So it's a sort of workaround? I was disappointed by that statement of his.
I don't get it. What can you write in RSpec that you can't write, nearly the same way, in Test::Unit?
He says Test::Unit is tightly coupled to the actual code and that that's a bad thing.
Done right, either one tightly _coheres_ to the actual code.
(Define "couple" as "A must change only because B changed", and define "cohere" as "A and B share legitimate reasons to change together".)
David Chelimsky wrote: >> I may be biased, but I would really like and have been looking for an >> alternative BDD choice besides RSpec. >> >> What others are there? I have seen a couple by the likes of Ryan, even >> Rick. Are any better? What about a conrast of testing libraries >> highlighting the benefits and opinions of each? >> >> I really want to get into testing beyond Test/Unit, but I detest the >> RSpec library out of spite
> > At the risk of shooting myself in the foot, I'm really curious to know > why you detest RSpec.
I'd like to chime in. I would not at all say that I detest RSpec in any way, but I do find it to be rather besides the point. It just never caught on to me, you know?
When I first looked at RSpec, I thought it was rather cryptic. This boils down to a matter of taste, I know, but all those "recipes" (I don't know the term) really aren't all that legible to me. "bowling.score.should == 0"? "should" is equal to 0? Is it any more expressive than "assert 0, bowling.score"? Again, it's personal preference, but this is one DSL that I don't care for. So I dismissed it in favor of the Test::Units I have come to cherish.
Then I attended a lecture by Aslak Hellesøy. I mean, with so many people around raving about RSpec, I must be missing the point right? But I was let down: Alsak demonstrated it to be a wrapper DSL around Test::Unit for "people who find it hard to come to gripes with TDD". So it's a sort of workaround? I was disappointed by that statement of his.
He says Test::Unit is tightly coupled to the actual code and that that's a bad thing. I say I like that coupling, because it keeps a sharp edge on the test-before-you-write principle: if you're mucking around code or tests that worked before, you'll find yourself pressed to advocate that change and express it in an additional or revised test.
Moving from personal preference into the field of computer science, there is no argument that I know of favoring either in code coverage or testing metrics. (Indeed, "coupling" and "cohesion" have so far only referred to the functional components themselves, not their tests.) This doesn't prove either Alsak or me wrong or right, but does iterate that it's a matter of preference.
"But," some might say, "we're testing *behavior* here. That's different. We can clean up methods, throw code around, but as long as the behavior remains the same, then that's OK." I see it as the same discussion about white box testing and black box testing, but with a twist.
As a black box, the bowling example on the RSpec home page seems to work fine. As a white box, it's lacking in every respect because the algorithm we assume to be present is, in fact, completely absent. I like to test the algorithm, because if I don't prove the behavior of the algorithm to be correct then I will need to verify every possible outcome instead. That kind of verification isn't very efficient.
So the twist is this: this time around, we're mixing up black box testing by dubbing it "behavior" and white box testing by performing actual method calls. I've been searching for a rationale, but could only find the one by Alsak that I was disappointed by before.
The way I see it, RSpec is just a matter of preference, and a bit of a hype at that. Perhaps I'm misunderstanding, and if so, I'd love to see the entire mantra clarified. But until then, I'm preferring Test::Unit for its readability and code coupling, and saying nay to all those who preach that RSpec is a level beyond Test::Unit.
I'm sorry that you see this as preaching and mantras. I have no intention of converting you. I like RSpec because it expresses the things I want in a BDD framework. You don't have to like it.
BDD comes from TDD, which comes from XP, which includes Refactoring as an integral part of the process. Refactoring is more expensive as your tests are more tightly coupled to the code, so one goal of TDD is to decouple the tests from the detail of the code (which is what changes most often). One thing that aids in this decoupling is focusing on behaviour instead of internal implementation. BDD champions this goal by putting it front and center. RSpec supports this by trying to use words like "describe" instead of "TestCase" and "should" instead of "assert."
It is clear from your arguments that a) your testing goals do not align with those of RSpec and b) you've only read the cover (your argument about the example on the Home Page). If you're seriously interested in learning about RSpec and BDD, then you should try to actually use it on a small project and see how it feels. If not, then so be it. I still get to use it, so I am happy.
Cheers, David
Roderick: David Chelimsky might be a little miffed with your selective quoting which makes it appear as if he doesn't like RSpec...
David Chelimsky wrote: >> I may be biased, but I would really like and have been looking for an >> alternative BDD choice besides RSpec.
This was not written by David, but was written in response to an email from him. The only clue to this is the level of indentation of the quoting.
Actually, this makes me laugh more than piss me off. But thanks for
sticking up for me
Cheers, David
Would you be kind enough to briefly explain what you perceive to be the advantages to RSpec? I’m really hoping for a good comparison of the two, so I can see if it’s worth investigating further. (I am writing a book that will include testing, and need more info). I’ve played with RSPec and haven’t really found anything I can’t do with test:unit… so that’s why I’m looking for more info. I’d really appreciate some input. I suspect that it’s much like TDD… you have to do it before you really appreciate it.
I don't know that there is anything that you can't do with test/unit - just that rspec comes w/ a lot of this stuff out of the box. And I think it is something experiential. Many people who really enjoy using RSpec say they were skeptical at first, but have grown to love it.
Here are some specific comparisons:
- dsl for expressing/organizing examples using strings
describe Thing do it "should do cool stuff" do ... end end
vs
class ThingTest << Test::Unit::TestCase def test_should_do_cool_stuff ... end end
I find that this encourages better naming and organization.
- dsl for expressing expectations
team.should have(11).players
vs
assert_equal 11, team.players.size
- better failure messages
expected empty? to return true, got false
vs
expected true, got false
- better reporting
Thing - should do this - should do that - should do the other thing (FAILED - 1) - should do even this (PENDING: Not Yet Implemented)
vs
...............
There is also a nicely formatted HTML report out of the box:
http://rspec.rubyforge.org/report.html
- rspec_on_rails - component isolation allows you to test views before controllers or models even exist. This is VERY helpful when you're on an XP or XP-like team.
I'm sure there's more that I'm not thinking of right now. There's also the "joy" factor. I find it a more joyful experience to use RSpec, for the reasons stated above and for reasons that you can only appreciate by exploring it.
Hope this helps.
Cheers, David
As an aside, RSpec's use of "should" may be confusing to anyone coming from the networking world:
http://www.faqs.org/rfcs/rfc2119.html
-faisal
> RSpec supports this by trying to use words like "describe" instead > of "TestCase" and "should" instead of "assert."
As an aside, RSpec's use of "should" may be confusing to anyone coming from the networking world:
RFC 2119 - Key words for use in RFCs to Indicate Requirement Lev (RFC2119)
Can't clarify everything for everybody I guess.
Brian Hogan wrote:
Would you be kind enough to briefly explain what you perceive to be the advantages to RSpec? I'm really hoping for a good comparison of the two, so I can see if it's worth investigating further. (I am writing a book that will include testing, and need more info). I've played with RSPec and haven't really found anything I can't do with test:unit....
And that's as expected. As you'll read in most intros to BDD, it's the same practice as TDD, but with the focus shifted. Everything you can do with BDD can be done with TDD.
People seem to be expecting some sort of sea change in development practices, but that's not going to happen here. And from what I've read, it was never intended to.
It's a small tweak, to refocus TDD practitioners a slight bit. They think it produces better specs/tests, not because it's something that couldn't be done before, but because it's now more obvious what you should be doing.
If TDD has always worked well for you (as I've read before, "if you've been doing it right all along"), then great, stick with it.
But the very simple terminology changes are what made the point of TDD finally click for me.
As for rspec specifically? It does what I need it to, produces phenomenally legible specs, and then stays out of the way.
Kyle
Great stuff. If you were going to expose people to tests, would you start with TDD and test:unit, or would you go right to BDD and RSpec?
Green field? i.e. no experience with testing at all? Definitely RSpec.
Beyond that, it depends on myriad factors, mostly boiling down to what is going to be most pragmatic for the team based on their level of experience with Ruby, testing in general, other frameworks, etc. I'd always prefer RSpec because, well, I prefer it. But sometimes going w/ test/unit could be more pragmatic, at least in the short run.
David
This argument is the same one you could use to say “what can you write in Ruby that you can’t write, nearly the same way, in C”? Heck, they are both TC general purpose computer programming languages. Ahhhh, but the “feel” is different. That’s what I find sets RSpec apart, and if it doesn’t click for you it might not work for you.
s.ross wrote:
This argument is the same one you could use to say "what can you write in Ruby that you can't write, nearly the same way, in C"?
Uh, in the same volume of code? At the same velocity?
OK, so what about python?
I think the point Steve was making was about "feel", which is what Ruby and Rails are largely about. To me, there is a huge difference between this:
team.should have(11).players
and this
assert_equal 11, team.players.size
And an even bigger difference in the output when these fail:
expected 11 players, got 10
vs
expected 11, got 10
Implementing these probably takes the same amount of time. A month later, seeing "expected 11 players" is VERY meaningful, especially when multiple failures appear due to a change you just made. If I see 10 failures that all say "expected x things, got y", then I have a lot more information right away about what went wrong than if I just see "expected x, got y".
FWIW.
David
I'd like to chime in. I would not at all say that I detest RSpec in any way, but I do find it to be rather besides the point. It just never caught on to me, you know?
When I first looked at RSpec, I thought it was rather cryptic. This boils down to a matter of taste, I know, but all those "recipes" (I don't know the term) really aren't all that legible to me. "bowling.score.should == 0"? "should" is equal to 0? Is it any more expressive than "assert 0, bowling.score"? Again, it's personal preference, but this is one DSL that I don't care for. So I dismissed it in favor of the Test::Units I have come to cherish.
Cryptic? Seriously? "bowling score should equal 0" vs "assert 0 bowling score"
The first statement is parseable by a second grader. The second statement only makes sense if you are very familiar with test::unit.
I think the problem is that you're looking at the code and ignoring the meaning. You ask "'should' is equal to 0?" It seems that you're examining the code the way the interpretter would, and thinking, "the object returned by the method should is equal to 0." That's completely the wrong mindset.
I would say that TDD is a tool to help you solve the problem of designing and implementing behavior. Test::Unit works fine in that regard, but RSpec reduces the semantic distance between the developer and the problem domain.
Consider a task list you write for yourself. Again you'll have
statements that look like "bowling score should equal 0" rather than
"assert 0 bowling score." Throw in a couple dots and you're good to
go RSpec lets you express behavior in code almost as neatly as you
do in your head. That's very, very powerful. I've yet to come across
a tool that brings you that close.
Then I attended a lecture by Aslak Hellesøy. I mean, with so many people around raving about RSpec, I must be missing the point right? But I was let down: Alsak demonstrated it to be a wrapper DSL around Test::Unit for "people who find it hard to come to gripes with TDD". So it's a sort of workaround? I was disappointed by that statement of his.
I'm not sure how long ago that lecture was, but to the best of my
knowledge RSpec is no longer just a wrapper around Test::Unit. On the
other hand, I'm not sure why it matters how it's implemented under the
hood. We are concerned with behavior here, after all 
Anyway, the big BDD guys all basically say that there's nothing different if you're doing TDD right. BDD/RSpec isn't a workaround to TDD, but rather a fast track to the good things about TDD. Again you have the benefit of a shorter semantic distance. By using RSpec you start off focusing on design and behavior, which is the ultimate goal of TDD.
He says Test::Unit is tightly coupled to the actual code and that that's a bad thing. I say I like that coupling, because it keeps a sharp edge on the test-before-you-write principle: if you're mucking around code or tests that worked before, you'll find yourself pressed to advocate that change and express it in an additional or revised test.
The concern is that Test::Unit encourages an artificial coupling of one test class to one production class. That's a completely different thing from tests being coupled to the code, which as you pointed out will/should be the case. The problem is that a new test class can feel heavy weight (especially if you've been using JUnit for several years, you almost certainly have that stigma). People think, "wow why should I create a whole new class when it's just a little bit of behavior?" Most people know that it's not a good idea to have a class with just one or two methods, but testing has different rules and the fact is that a test class with just a few methods is often precisely what you need.
RSpec helps you avoid this problem by encouraging you to write several small, focused examples. Once again it's not something that you couldn't do with Test::Unit, but the RSpec way is just a lot closer to the Right Way.
Moving from personal preference into the field of computer science, there is no argument that I know of favoring either in code coverage or testing metrics. (Indeed, "coupling" and "cohesion" have so far only referred to the functional components themselves, not their tests.) This doesn't prove either Alsak or me wrong or right, but does iterate that it's a matter of preference.
"But," some might say, "we're testing *behavior* here. That's different. We can clean up methods, throw code around, but as long as the behavior remains the same, then that's OK." I see it as the same discussion about white box testing and black box testing, but with a twist.
As a black box, the bowling example on the RSpec home page seems to work fine. As a white box, it's lacking in every respect because the algorithm we assume to be present is, in fact, completely absent. I like to test the algorithm, because if I don't prove the behavior of the algorithm to be correct then I will need to verify every possible outcome instead. That kind of verification isn't very efficient.
I don't think RSpec is meant to be a white-box testing tool. Again, it's all about design and behavior. Despite that, you should still be able to achieve a sufficient level of white-box testing using mocks.
In fact, sometimes I'll write two sets of specs for the same example. One will use concrete implementations, allowing me to verify that I get the desired side-effects (records created, etc). Another one will use mocks, allowing me to verify that the proper interactions are being made. I've got a good example, but I don't want to add 65 lines of code to this already lengthy message...but if there's any interest I'll be happy to post it in a followup message.
So the twist is this: this time around, we're mixing up black box testing by dubbing it "behavior" and white box testing by performing actual method calls. I've been searching for a rationale, but could only find the one by Alsak that I was disappointed by before.
The way I see it, RSpec is just a matter of preference, and a bit of a hype at that. Perhaps I'm misunderstanding, and if so, I'd love to see the entire mantra clarified. But until then, I'm preferring Test::Unit for its readability and code coupling, and saying nay to all those who preach that RSpec is a level beyond Test::Unit.
Sure, just as some people prefer C. It's certainly more readable - you have semicolons that tell you where a statement ends!
Pat
We could debate either side of the issue. I could say “with the same compactness? with equivalent performance characteristics?” but at the end of the discussion, it’s all about what clicks for you. If you are happy with asserts (like in C, hence my analogy), then use them. I believe the BDD people – and I count David among one of the most thoughtful of the bunch – have put a good deal of work into making the description of the spec as natural as possible. Again, it’s what reads naturally to you. If you don’t parse it naturally, then no prob. Also, make careful note of what David says about the failure messages. This was one of my biggest complaints with Test::Unit and the fix is pretty close to free with RSpec. The failures report pretty much in English and if they don’t I probably wrote the example wrong.
–s
Pat Maddox wrote:
In fact, sometimes I'll write two sets of specs for the same example. One will use concrete implementations, allowing me to verify that I get the desired side-effects (records created, etc). Another one will use mocks, allowing me to verify that the proper interactions are being made. I've got a good example, but I don't want to add 65 lines of code to this already lengthy message...but if there's any interest I'll be happy to post it in a followup message.
This sounds very interesting. I would love to see a concrete example detailing your approach.