why detest rspec? (was: community poll about testing and specs)

Okay, here we go :slight_smile:

describe VideoService, " when creating a video" do   fixtures :companies, :sites, :video_formats, :publish_settings

  before(:each) do     @video = Video.new     @video.company = companies(:test)     @video.site = sites(:test)     sites(:test).stub!(:formats).and_return [ video_formats(:flash) ]

    @origin = mock_model(Origin, :process_source => true, :transcode_asset => true, :storage => "file")     Origin.stub!(:find_transcoder).and_return @origin

    @service = VideoService.new   end

  def create_video     @service.create_video :video => @video, :signature => "abc123", :frame => 1, :storage => "file"   end

  it "should add a new video record" do     lambda { create_video }.should change(Video, :count).by(1)   end

  it "should create two assets - one source and one transcoded" do     lambda { create_video }.should change(Asset, :count).by(2)     @video.source_asset.should be_source     @video.should have(2).assets   end

  it "should create the main asset with the appropriate video format" do     create_video     @video.assets.find_by_video_format_id(video_formats(:flash).id).should_not be_blank   end end

describe VideoService, " when creating a video" do   fixtures :companies, :sites, :video_formats

  before(:each) do     @video = mock_model(Video, :save => true, :company_id => companies(:test).id, :site => sites(:test), :hook => nil)     sites(:test).stub!(:formats).and_return [ video_formats(:flash) ]

    @source_asset = mock_model(Asset, :video_id= => @video.id, :save => true)     @transcode_asset = mock_model(Asset, :video_id= => @video.id, :save => true)     Asset.stub!(:new).and_return @source_asset, @transcode_asset     @video.stub!(:source_asset).and_return @source_asset

    @source_origin = mock_model(Origin, :process_source => true, :transcode_asset => true, :storage => "file")     @transcode_origin = mock_model(Origin, :transcode_asset => true, :storage => "file")     Origin.stub!(:find_transcoder).and_return @source_origin, @transcode_origin

    @service = VideoService.new   end

  def create_video     @service.create_video :video => @video, :signature => "abc123", :frame => 1, :storage => "file"   end

  it "should find a transcoder" do     Origin.should_receive(:find_transcoder).with(companies(:test).id, false, "file").and_return @source_origin     create_video   end

  it "should process the source asset" do     @source_origin.should_receive(:process_source).with("abc123", @source_asset, 1, "http://test.host:80/"\)     create_video   end

  it "should find an origin for the transcode format" do     Origin.should_receive(:find_transcoder).with(companies(:test).id, false, nil).and_return @transcode_origin     create_video   end

  it "should transcode the new asset" do     @transcode_origin.should_receive(:transcode_asset).with("abc123", @transcode_asset, "http://test.host:80/", "http://test.host:80/video_formats/#\{video\_formats\(:flash\)\.id\}\.xml"\)     create_video   end end

I created the VideoService to wrap the creation of a video in our system. At that stage in the code, there were a couple main parts - creating a new video record, creating a couple asset records, and notifying our transcode machines about the new assets.

The first example specifies the db behavior I want. That was my black-box test (you might say it's grey-box because I'm stubbing some stuff out, but that wasn't part of the original spec. It's just there so the spec can run - Origin makes a network call and it's easier to stub the whole thing out).

The second example is the white-box test, which verifies that the internals work. Certain origins should be found, they need to receive certain calls with a particular video, etc.

Now I'm not sure if people will like those specs :slight_smile: but hopefully it gives you an idea of how you can approach specifications from both angles.

Pat

Hi --

Fortunately, if you find "should" confusing or misleading you can write your examples such as this one:

  it "should redirect to the home page after logging out"

like this instead:

  it "must redirect to the home page after logging out"

Or using any other word or language you think is better. The language you employ in your examples is up to you, although the RSpec community does tend to use some widespread patterns, including the use of the word "should" in a way that is closer to the "standard English sense" rather than the "RFC-2119 sense".

Cheers, Wincent

I'm pretty sure that he's referring to foo.should == bar

However I have to say I think it's an absurdly nitty "issue"

Pat

My perception is based on your statement that you find the tight coupling between tests and the subject code to be a good thing, and that you like the fact that making changes to code forces you to make changes to your tests.

Compare this to TDD's goal of supporting inexpensive refactoring through very loose coupling between tests and code.

I also sensed that you don't find value in focusing the discussion on behaviour:

- better reporting

Thing - should do this - should do that - should do the other thing (FAILED - 1) - should do even this (PENDING: Not Yet Implemented)

vs

...............

I first saw RSpec in one of the tutorials at RailsConf this year. The positive reinforcement from seeing what you were testing each time was compelling. I also find seeing the list of tested behaviors often suggests to me the next test/feature that needs to be added to the project. (Now if I could just get my rspec.conf file to be used so I don't have to pass '-f specdoc' each time I run my tests.

- rspec_on_rails - component isolation allows you to test views before controllers or models even exist. This is VERY helpful when you're on an XP or XP-like team.

David, could you expand on this? Or post a link to docs or a tutorial or blog post about testing views with RSpec.

Look here:

http://rspec.rubyforge.org/documentation/rails/writing/views.html

The idea is that you can mock enough to populate the view data allowing you to design from the "outside in." Basically mock up your UI, do it in RHTML (or whatever markup you like), and write specs for that.

Pat Maddox wrote:

> > > > > > RSpec supports this by trying to use words like "describe" instead > > > of "TestCase" and "should" instead of "assert." > > > > As an aside, RSpec's use of "should" may be confusing to anyone > > coming from the networking world: > > > > RFC 2119 - Key words for use in RFCs to Indicate Requirement Lev (RFC2119) > > Fortunately, if you find "should" confusing or misleading you can > write your examples such as this one: > > it "should redirect to the home page after logging out" > > like this instead: > > it "must redirect to the home page after logging out" > > Or using any other word or language you think is better. The language > you employ in your examples is up to you, although the RSpec community > does tend to use some widespread patterns, including the use of the > word "should" in a way that is closer to the "standard English sense" > rather than the "RFC-2119 sense". > > Cheers, > Wincent

I'm pretty sure that he's referring to foo.should == bar

Ah, yes, you're almost certainly right.

Well the solution to that is also only a step away (alias "should" and "should_not" to whatever you want in your spec helper file). Of course, I wouldn't actually recommend doing so, as it seems to be going against the grain for no real good reason.

Cheers, Wincent

Brian Hogan wrote:

@David, @Kyle:

Great stuff. If you were going to expose people to tests, would you start with TDD and test:unit, or would you go right to BDD and RSpec?

Our team adopted RSpec about a year ago and we've been through several projects entirely without Test::Unit. If someone has never had any experience with TDD, then I would advocate learning through BDD... and would recommend RSpec as the framework to learn it with.

On a side note. JRuby 1.0 will come with two gems.

Rake and RSpec.

Yes... people are using it in the real world and I can testify on it's behalf in court. It's elegant. Our developers *LOVE* it. I love it. Even our Interaction Designers have picked up on the lingo and use "should"-style language in their interface specifications, which translates nicely into our implementation process.

RSpec ftw!

Robby

s.ross wrote:

This argument is the same one you could use to say "what can you write in Ruby that you can't write, nearly the same way, in C"? Heck, they are both TC general purpose computer programming languages. Ahhhh, but the "feel" is different. That's what I find sets RSpec apart, and if it doesn't click for you it might not *work* for you.

Agreed. I also consider RSpec to be one of those most exciting things to pop up in the Ruby community since Rails. Our pure-Ruby libs are being spec'd (not tested) and we can pretty much run rm -rf test/ in our Rails applications.

It might not sit well with all, but the syntax is a great example of how great and expressive Ruby can be.

Robby

Roderick van Domburg wrote:

David Chelimsky wrote:

To me, putting the focus on behaviour encourages a subtle, yet powerful change in how you perceive these things we call tests/specs/examples, etc. Obviously, RSpec is interested in helping you to think of tests as executable examples of expected behaviour as opposed to, well, tests.

This help?

Yes, that is a metaphor that sticks.

All that rests now is how "renaming" assert(x, y) to Object#should truly facilitates that idea. Yes there are other niceties and syntactic sugar, but isn't providing a humanized DSL really the gist of it all?

In the end, you'll still be comparing method results and variable assignments.

Yes, in the end... you see a result like this. But, I'd like to back this up to before you write your code.

Let's start with a high-level business rule for an application. We might have a discussion with our client and determine the following.

A user should not be allowed to create a new account when providing an existing email address.

This is something that we'd have our clients agree upon before we ever touched any code. When it comes time to implement this behavior into the application, we can translate this into RSpec very easily.

describe User, "new account" do

   it "should not be allowed to create a new account when providing an existing email address."

end

When we run our specs, this will show up as unimplemented. I often add several of these once a client has signed off on some specifications, and it results in a TODO-list for me. At this point, I can begin turning these into full specs.

In a nuthsell, it's not just a DSL for Ruby... it's provides a framework for discussing requirements with our clients and Interaction Design team. It's clear English and leaves little room for ambiguity, which I believe is a good thing! :slight_smile:

If you enjoy translating high-level requirements into Test::Unit... and don't mind the underscore/camelcase madeness... well, good luck with that. I switched to Ruby/Rails because of how expressive and English-like it was and think that it's very fitting that RSpec would evolve out of this community.

Cheers, Robby

Hi --

This is something that we'd have our clients agree upon before we ever touched any code. When it comes time to implement this behavior into the application, we can translate this into RSpec very easily.

describe User, "new account" do

  it "should not be allowed to create a new account when providing an existing email address." end

When we run our specs, this will show up as unimplemented. I often add several of these once a client has signed off on some specifications, and it results in a TODO-list for me. At this point, I can begin turning these into full specs.

In a nuthsell, it's not just a DSL for Ruby... it's provides a framework for discussing requirements with our clients and Interaction Design team. It's clear English and leaves little room for ambiguity, which I believe is a good thing! :slight_smile:

If you enjoy translating high-level requirements into Test::Unit... and don't mind the underscore/camelcase madeness... well, good luck with that. I switched to Ruby/Rails because of how expressive and English-like it was and think that it's very fitting that RSpec would evolve out of this community.

OK... but, lest we forget, Nathaniel's work on test/unit -- past, present, and future (I hope) -- has played an incalculable role in shaping the Ruby programming culture. Luckily, it's not a zero-sum game; people can delve into RSpec without having to jettison or scoff at test/unit. It's possible for two highly-accomplished test frameworks to evolve out of the same community :slight_smile:

(I'm not trying to get you to like test/unit; I'm just putting in a word for not having it have to be a winner-take-all kind of relationship, and also eager to keep the significance of test/unit to Ruby culture on the radar. I know I'm showing my age -- in this case, almost seven :slight_smile:

David

I think this is one of several aspects of RSpec that help promote this idea. I do thing that saying "should" instead of "assert" changes the feel from tests to examples of behaviour. But it's not limited to that.

The structure/organization using describe/it instead of TestCase/test_method and, especially, using strings to express the intent rather than method names both go a long way to support this shift in focus. The reporting you get helps as well.

David

Hi --

> This is something that we'd have our clients agree upon before we ever > touched any code. When it comes time to implement this behavior into the > application, we can translate this into RSpec very easily. > > describe User, "new account" do > > it "should not be allowed to create a new account when providing an > existing email address." > end > > When we run our specs, this will show up as unimplemented. I often add > several of these once a client has signed off on some specifications, > and it results in a TODO-list for me. At this point, I can begin turning > these into full specs. > > In a nuthsell, it's not just a DSL for Ruby... it's provides a framework > for discussing requirements with our clients and Interaction Design > team. It's clear English and leaves little room for ambiguity, which I > believe is a good thing! :slight_smile: > > If you enjoy translating high-level requirements into Test::Unit... and > don't mind the underscore/camelcase madeness... well, good luck with > that. I switched to Ruby/Rails because of how expressive and > English-like it was and think that it's very fitting that RSpec would > evolve out of this community.

OK... but, lest we forget, Nathaniel's work on test/unit -- past, present, and future (I hope) -- has played an incalculable role in shaping the Ruby programming culture. Luckily, it's not a zero-sum game; people can delve into RSpec without having to jettison or scoff at test/unit. It's possible for two highly-accomplished test frameworks to evolve out of the same community :slight_smile:

More than two. Don't forget watir. And all of the mocking frameworks. There's a lot of good stuff here.

(I'm not trying to get you to like test/unit; I'm just putting in a word for not having it have to be a winner-take-all kind of relationship, and also eager to keep the significance of test/unit to Ruby culture on the radar.

A very important point. And now that Ryan has taken over maintenance of test/unit, I expect to see some great new ideas appearing there as well.

The thing about RSpec is that it is intended to be a BDD framework, not a testing framework. It aims to support a behaviour-focused process in which you, in very granular steps, describe a bit of behaviour and then implement it.

But BDD is not the end of the testing story on any project. At least I don't think it _should_ be. Some might even say it's not part of the testing story at all :slight_smile:

A perfect example is Matrix Testing, introduced by ZenTest. In fairness, I haven't done this yet, but I can imagine a process that goes like this:

1. Develop a component using TDD (and RSpec) to encourage simple design and usable APIs. 2. Write a matrix test (using ZenTest with test/unit) to ensure that you've covered all of the permutations you anticipate. 3. If the matrix test exposes any holes, go back and use RSpec to drive the process of plugging them.

I can hear some saying "but that's doing the same work twice. Why not just a Matrix Test to begin with and call it a day?"

Because these two tools/approaches solve different problems. RSpec/BDD is good at expressing behaviour and encouraging simple, flexible designs. MatrixTesting is good at expressing multiple permutations in a clean and simple way. Used together, I believe that you'd end up with a simpler design with better test coverage than you would with either tool alone.

In XP we do Customer Tests and Developer Tests, though these have a lot of different names. They are, in the end, exercising the same code. But they express very different things. And what my experience tells me is that it's the combination of layers of testing that proves most effective. So for me, this notion of combining a BDD framework (for DESIGN) with a testing framework (for TESTING) makes perfect sense.

I know I'm showing my age -- in this case, almost seven :slight_smile:

Happy Birthday (6 year olds only say "almost 7" when it's just a few days away).

Cheers, David

http://blog.davidchelimsky.net/articles/2006/11/06/view-spec-tutorial http://rspec.rubyforge.org/documentation/rails/writing/views.html http://rspec.rubyforge.org/rdoc-rails/index.html

Cheers, David