Brainwashing needed

Hello!

Could someone point me to a convincing document about rails testing?

My problem is this:

Testing is important in rails. But I really don't understand why.

I've read core members rejecting patches.
I've read HOW to make tests.
I've read the rails testing manual.
I've read agile web dev section on testing.

I like to write pretty and understandable code, documentation, comment
my code well, but I'm coming from a PHP background and there is
something about rails testing I just don't understand - what is the
real benefit for writing tests on a web application?

I don't want to be the blind sheep and follow the mantras without
understanding. But I don't want to fail to grasp an integral part to
working with this wonderful useful tool called rails.

Thanks!

I can give you some anecdotal evidence about why testing is a good thing to do, and one reason why I’m a believer in Test Driven Development. I have written a large-ish e-commerce / fulfillment application (4,500+ lines of code, 6,000+ lines of test code), and having the tests in place has saved me hours of work and potentially lost revenue due to subtle bugs.

For example, at one point I was working on adding a new feature to the application after it was in a fully-working state with tests covering much of the functionality. I added new tests for the new functionality, added the new functionality, and ran the new tests (via rake recent). All the new tests passed and I thought I was good to go. Then I ran all the tests, and discovered I had just created a new bug in a completely separate part of the application (the new code I had added was deep in the area of cost calculations). I am sure I would not have discovered that newly-created bug in a reasonable amount of time if I had not had the tests in place to validate the performance of my application.

So, there’s at least one example of automated unit tests saving a lowly developer’s back side. :slight_smile:

The real benefit of testing won’t be seen (in my opinion) unless you write your tests first.

For example… here’s a quick unit test for a user model.

def test_generate_new_password
u = User.find_by_username
“homer”
old_password = u.password_hash
newpass = u.generate_new_password(8)

 u = User.find_by_username "homer"
 assert old_password != u.password_password_hash

 assert User.try_to_login("homer", newpass)

end

I’m writing this FIRST, before i have done anything in the model. I know what I want my user model to do and I know what the outcome should be. Once I have the test written, I can go into the model and actually write the code to make my test pass. When my test does pass, I can move on to the next business rule, knowing that when it comes time to build my pages, I won’t be sitting around debugging the whole stack when the password reset feature doesn’t work.

Here’s another example: I want the user to be redirected to the login page if they are not logged in. I will write an integration test.

def test_login_redirection
get “/admin/index”
assert_response :redirect

follow_redirect
assert_response :success
assert_template "login"

end

Now I just have to put in the appropriate before filters in my application to make this test pass. I can add to this test later to ensure that a successful login brings the user back to the page they initially requested.

All of this really helps when you are working on the project and the business rules change. You can use your tests to see what you have to change or modify. It’s also very useful in large team projects, where your tests will prove that your change doesn’t break anyone else’s work. That’s why the Rails core requires them with patches.

Hope that helps!

Do a google search and you'll find many reasons why people write unit
tests: I believe for most people, it gives peace of mind knowing if you
change code, you don't break existing functionality. Think about it,
you have to test your code anyway so why don't you write it down in
executable form (and Rails makes it so easy) and it'll save you many
hours later. I'd like to think of it as "reusable testing".

TDD takes it a step further and help to guide you to a better design.
I'd recommend reading Kent Beck's "Test Driven Development: By Example"
[1].

[1]
http://www.amazon.com/Test-Driven-Development-Addison-Wesley-Signature/dp/0321146530

Remember though that tests need to be changed. I’ve worked on projects where the tests just get stale. That peace of mind only holds up if your developers keep those tests in good shape, and not just altering them so they pass.

Here is another example of why testing is just all around goodness. I
find it especially useful for those "oh damn" moments when your
writting an application. What I mean by that, is that moment in a not
so greatly planned project where you realize one of your fundemental
classes or building blocks of code needs to be changed (some might say
refactored) and you know that change is going to affect lots. I just
recently ran into this. Normal I would have dreaded going back and
trying to stomp out the bugs and such. However since I already had
tests in place, I just fired them off and immediatly I new what had
broken in my app, and why.

It would have taken days, maybe even weeks to change directions in my
project with out the tests being in place, finding bugs and fixing
them. With the tests, it took about 3 hours.

Well thats what coverage tools do for you. While coverage isn't everything,
it is at least a metric that can drive test quality when used responsibly.

I have no experience with rcov (will change soon I expect) but Clover in Java
(a commercial product, but not expensive) was excellent.

We have seen problems with Developers deleting tests - the hostility this
can create is immense.

Test-first development seems a bit easier in some languages like Ruby. In
Java this seems a bit less popular - your tests won't compile if the classes
under test do not exist yet, so TFD is a bit inelegant at the outset.

There is another side effect of TDD - over time the Tests document your
code. They can provide a good learning model for introducing team
members. Also TFD can provide a good model for working with remote
sites where implementation & design are done by different people/groups.

Basically you write a test that exercises a coding contract (the implementation)
that you want the other team to fulfill.

Unit Tests own. While networked stuff is a bit harder to unit test, there is no
excuse for not using them in other domains. And yes, Rails makes it easy,
even for the networked/DB stuff.

@Richard

“Test-first development seems a bit easier in some languages like Ruby. In
Java this seems a bit less popular - your tests won’t compile if the classes
under test do not exist yet, so TFD is a bit inelegant at the outset.”

Exactly. That’s part of test-driven development. Tests not compiling is failing :slight_smile:

You will like rcov. It’s quite handy although not perfect. Combine that with ZenTest and you have no excuse NOT to do unit testing.

Absolutely. Anything when not done appropriately will not bring much
benefit (even become problems). In this case, unit testing can give a
false sense of security if not done properly.

Hey thanks everyone for your reply! My post didn't make it to the list
24 hours after I posted it, so I didn't get a chance to read your
responses to now.

I think I'm in a position that must be common: Coming from PHP,
extremely attracted to rails, experimenting in my free time, writing
small pet apps. Until recently testing was "that thing I'll learn after
I get the basics down" - in other words, it's more important to learn
how ruby works, how rails works....and at some point I turned around
and said "Wait, testing, I forgot about that whole bit"

It sounds like I just need to shut up and do it, and see what the
benefits are like. It sounds like the biggest benefit is catching nasty
buried bugs as the app is progressing. As I'm not building commercial
apps with rails right now, it seems easier to ignore testing during the
experimentation process - hence a 'bad habit' is in process of forming!

@Richard

"Test-first development seems a bit easier in some languages like Ruby. In
Java this seems a bit less popular - your tests won't compile if the classes
under test do not exist yet, so TFD is a bit inelegant at the outset."

Exactly. That's part of test-driven development. Tests not compiling is
failing :slight_smile:

Yeah but the stack-trace bomb is a pretty inelegant part of failing. I would
like it to gracefully fail with error messages, and not play havoc with the
intellisense in my IDE either, making my source code look like a letter
a six-year-old typed out with their nose.

But thats just a statically-typed thing that makes Test-first development
a bit kludgy to start with. However it is a significant part of why TFD is
not done. For continuous integration systems, I would prefer if tests
failed during the unit-test phase and not during code compilation.

(okay the fact that I checked in uncompiling code into a CI system is
a fudged example, but you can see where I am coming from). Still
you can see that this is a pretty Java (presumably any other statically
typed/or compiled language problem too) test idiosyncrasy, a Ruby
unit test would fail during the test phase of CI just where it should.

TFD is way more powerful than Test-second-Design. Both are good, but
you will get more churn in Test-second-Design. Over time Test-second
design forces an implementation that you would have gotten to earlier
with TFD. TFD is better at promoting KISS as it makes your tests easier
to implement. So TFD enforces better habits.

I have found that Test Driven Development (any kind) makes me think
differently about the code under development. If I am stuck in a coding
problem (paralyzed by implementation choices) I can go into unit test
mode and it often steers me through this process.

Also TDD is the mongoose to the snake of obtuse design. If it is hard
to unit test an obscure object hierarchy, then you have to give up testing
or unravel the implementation. Your colleagues would appreciate the
latter a lot. Java development contains a lot more fashionably-obtuse class
hierarchies than is healthy.

You will like rcov. It's quite handy although not perfect. Combine that with
ZenTest and you have no excuse NOT to do unit testing.

I *loved* clover, if its anything close, I am sold. When you use a good coverage
tool to patch up a blind spot in your unit test harness and in doing so, locate
a bug, you are sold on it forever. Boosting coverage is a great way to consume
idle developer cycles or get new team members familiar with the code base.

@Jeff

Then you had month after grueling month of fixing bugs that were found
by the test group. This time, the testing/bug fixing stage had more to
do with working out the poorly worded parts of the Spec, rather than
fixing malfunctioning code. The code wasn't perfect (because the tests
weren't perfect), but the difference was night and day.

Note: Unit Testing can't find all bugs, even at 100% coverage. What it does
do is cover you for significant bugs that are tedious to test for.

Richard Conroy wrote:

@Richard

"Test-first development seems a bit easier in some languages like Ruby. In
Java this seems a bit less popular - your tests won't compile if the classes
under test do not exist yet, so TFD is a bit inelegant at the outset."

Exactly. That's part of test-driven development. Tests not compiling is
failing :slight_smile:

Yeah but the stack-trace bomb is a pretty inelegant part of failing. I would
like it to gracefully fail with error messages, and not play havoc with the
intellisense in my IDE either, making my source code look like a letter
a six-year-old typed out with their nose.

The advice I had from a very experienced Agile coach was to use the intelligent fixing capability in the IDE (Eclipse, in this case) to keep the code compiling and allow the failing test to run. There should be a TODO comment in any method that Eclipse has introduced for you, so the Tasks view gives easy navigation to these methods.

regards

   Justin Forder

That's a great example.

However, a lot of times I see many redundant test cases. Something like,

class Product < AR
  validates_presence_of :name
end

class ProductTest < TU
  def test_absence_of_name
   ..
   product.name = nil
   assert !product.valid?
   assert product.errors.invalid?(:name)
  end
end

This is more of testing ActiveRecord rather than testing your
application/business logic. I feel this is completely redundant.

Why? If you want to make sure your application is behaving as you wan't
it to you better test for it. Testing that AR is behaving as expected is
not much different from testing whatever other logic your application
might contain.

Perhaps you didn't quite understand totally how AR works when you
implemented the check. Perhaps you made a mishap without realizing it
because it was late. Without a test to catch it you wouldn't know.
Unittesting is a way to make sure the code behaves as expected and keeps
doing so.

If you don't want to test the above, why would you test an if statement,
I mean that's plain Ruby and is most likely to never fail on you. But
you write the test anyways, right? Again, you do it to make sure your
code is behaving as you intend it to.

In an ideal world it would be impossible to alter any part of an
application without some test breaking. That's what I strive for
anyways.

Why?

Mainly because of
http://dev.rubyonrails.org/browser/trunk/activerecord/test and
sticking to Rails' DRY principle.

If you want to make sure your application is behaving as you wan't
it to you better test for it. Testing that AR is behaving as expected is
not much different from testing whatever other logic your application
might contain.

It is different when you test rails's functions. It's same as any
other kind of testing when you have implemented a lots of callback
functions. But the example I gave you, doesn't make any sense where
your model doesn't have any callbacks.

If you don't want to test the above, why would you test an if statement,
I mean that's plain Ruby and is most likely to never fail on you. But
you write the test anyways, right? Again, you do it to make sure your
code is behaving as you intend it to.

Yeah. But we don't test each and every line of code. Rather, we test
the application logic. And we write test cases for code we wrote.

It's like writing functional test for following action :

def donttestme
redirect_to :index
end

And test if redirection is working.

I guess we all follow a general rule. Rails gives us the bugfree code
( forgot the edge hippies :wink: ) and we should blindly assume that when
we ask rails to check presence of a name, it's being checked.

It is different when you test rails's functions. It's same as any
other kind of testing when you have implemented a lots of callback
functions. But the example I gave you, doesn't make any sense where
your model doesn't have any callbacks.

You're not only testing that Rails own functions works as advertised.
You're also testing your usage of them. Which in my opinion is pretty
valuable.

> If you don't want to test the above, why would you test an if statement,
> I mean that's plain Ruby and is most likely to never fail on you. But
> you write the test anyways, right? Again, you do it to make sure your
> code is behaving as you intend it to.

Yeah. But we don't test each and every line of code.

So how do you know that the lines not tested are working as they should?

It's like writing functional test for following action :

def donttestme
redirect_to :index
end

And test if redirection is working.

And test that it redirects to index. Which if the test is written
properly would be quite nice to know when someday somebody decides that
index should be renamed to big_ol_johns_cup_of_dough.

The point being - as I tried to indicate in my previous post - is that
Rails might be working as it should, but you don't know if your usage of
Rails is correct unless you test and therefore you don't know if your
code is working as you want it to unless you test. It doesn't really
matter how trivial or complex your own code is.

Bugs don't need more than one line to hide in :wink:

Yeah. You're right.

Thanks,
Pratik

It was a nice thread to read. I think I should seriously use TDD approach in all of my projects.

Anil