Hi I have a named scope that sets a sort order. In my unit test I can run the query on a set of records (using Factory or fixtures) and check the sort order is correct. I believe though that a find without an explicit sort order may return the records in any order, so how can I be sure that it is not just accidental that they are in the correct order?
Colin
I have a named scope that sets a sort order. In my unit test I can run the query on a set of records (using Factory or fixtures) and check the sort order is correct. I believe though that a find without an explicit sort order may return the records in any order, so how can I be sure that it is not just accidental that they are in the correct order?
Colin:
I would argue that it's unimportant -- even at the unit-level -- *how* the test is passing. If every time you run the test it passes, then the code does what you specify it does. When there's a problem, identify the bug and write a new unit, integration, or functional test that eliminates the unexpected behavior, then make that test pass.
I might also argue that it's not up to you to test ActiveRecord. You might assert that your named scope is passing the correct sorting parameters to ActiveRecord; at that point, it's up to ActiveRecord to hold up its end of the bargain.
What do you think of that?
Andrew M. Kasper
I'd add that something that may give you some comfort is to endeavour to write a failing test first.
Fred
Colin Law wrote in post #965435:
Hi I have a named scope that sets a sort order. In my unit test I can run the query on a set of records (using Factory or fixtures) and check the sort order is correct. I believe though that a find without an explicit sort order may return the records in any order, so how can I be sure that it is not just accidental that they are in the correct order?
Try creating the factories in a weird order. If they still are returned in the correct order, then your sorting is correct.
...or...
Write the test so that it checks that the appropriate :order clause is set. This is probably too invasive and implementation-dependent for my taste, though I might do it occasionally.
Colin
Best,
I have a named scope that sets a sort order. In my unit test I can run the query on a set of records (using Factory or fixtures) and check the sort order is correct. I believe though that a find without an explicit sort order may return the records in any order, so how can I be sure that it is not just accidental that they are in the correct order?
I would argue that it's unimportant -- even at the unit-level -- *how* the test is passing. If every time you run the test it passes, then the code does what you specify it does. When there's a problem, identify the bug and write a new unit, integration, or functional test that eliminates the unexpected behavior, then make that test pass.
I don't think that is a valid argument. I think effectively what you are saying is that it doesn't matter if it actually tests what it is supposed to as long as it passes. If you later discover that the app is not doing what it is supposed to then add more tests.
I might also argue that it's not up to you to test ActiveRecord. You might assert that your named scope is passing the correct sorting parameters to ActiveRecord; at that point, it's up to ActiveRecord to hold up its end of the bargain.
I am not trying to test that ActiveRecord sorts correctly, I am trying to test that I have correctly coded the scope. I don't know how I would assert that the named scope is passing the correct sorting params. Can you enlighten me?
I think what I will do is use some common sense rather than being too pedantic about the testing. By creating a number of records in a random order and then checking that they are sorted correctly I think the possibility of accidentally finding them come back in sorted order is negligible. In addition I will have checked that the test fails before I add the sort clause.
Colin
Absolutely, I will check that the test fails before adding the order clause, but see my reply to Marnen below.
Colin
Colin Law wrote in post #965435:
Hi I have a named scope that sets a sort order. In my unit test I can run the query on a set of records (using Factory or fixtures) and check the sort order is correct. I believe though that a find without an explicit sort order may return the records in any order, so how can I be sure that it is not just accidental that they are in the correct order?
Try creating the factories in a weird order. If they still are returned in the correct order, then your sorting is correct.
Probably. I may be wrong but I believe that postgreSQL does not even guarantee that an unordered query will produce the same results if the same query is run twice. Conceivably therefore I could test without the order clause and check it fails, then, thinking I had added the order clause re-run the test and find it pass. I think though as I said in an earlier post that I can use common sense and assume the probability of this is so small as to be insignificant.
...or...
Write the test so that it checks that the appropriate :order clause is set. This is probably too invasive and implementation-dependent for my taste, though I might do it occasionally.
I agree this seems like overkill and would likely be fragile.
Colin
Colin Law wrote in post #965538:
in the correct order, then your sorting is correct.
Probably. I may be wrong but I believe that postgreSQL does not even guarantee that an unordered query will produce the same results if the same query is run twice.
Right, SQL databases cannot be assumed to guarantee this. That's why I suggested creating the factories in an arbitrary order, different from what you want to see.
Conceivably therefore I could test without the order clause and check it fails, then, thinking I had added the order clause re-run the test and find it pass. I think though as I said in an earlier post that I can use common sense and assume the probability of this is so small as to be insignificant.
Well, if you create the factories in order, then there's a significant probability of this happening...
Best,
Colin Law wrote in post #965538:
in the correct order, then your sorting is correct.
Probably. I may be wrong but I believe that postgreSQL does not even guarantee that an unordered query will produce the same results if the same query is run twice.
Right, SQL databases cannot be assumed to guarantee this. That's why I suggested creating the factories in an arbitrary order, different from what you want to see.
I don't think I made my point clearly. I believe that, in theory, even by creating the objects in an arbitrary order one cannot guarantee that an un-ordered query will not coincidentally end up with an ordered set. I do also believe however that the probability of this is so small as to considered negligible. It is still an interesting academic point if not an issue in practice.
Conceivably therefore I could test without the order clause and check it fails, then, thinking I had added the order clause re-run the test and find it pass. I think though as I said in an earlier post that I can use common sense and assume the probability of this is so small as to be insignificant.
Well, if you create the factories in order, then there's a significant probability of this happening...
Certainly.
Colin
Not that I think that this is the best way, actually, thats probably the worst, but on the same time, it is the easier:
/ORDER BY column/ =~ Model.your_scope.to_sql
I had not thought of doing something like that. Thanks
Colin
> I would argue that it's unimportant -- even at the unit-level -- *how* > the test is passing. If every time you run the test it passes, then > the code does what you specify it does. When there's a problem, > identify the bug and write a new unit, integration, or functional test > that eliminates the unexpected behavior, then make that test pass.
I think effectively what you are saying is that it doesn't matter if it actually tests what it is supposed to as long as it passes. If you later discover that the app is not doing what it is supposed to then add more tests.
A good point. I would summarize it more like this: "If you verify, to the best of your ability, that the software behaves the way you want it to, then you're finished testing until you discover an exception. When you have a repeatable case where the software does not behave the way you expect, at that point you will have steps to cause the unexpected behavior (and therefore a test)."
It's not *only* the case that the test needs to pass; as you mentioned, it is also the case that the test needs to verify behavior. Sounds like we definitely agree on that. And you're right; it's that "to the best of your ability" that's tricky. Seems like that's where we went down different paths.
> ...it's up to ActiveRecord to > hold up its end of the bargain.
I am not trying to test that ActiveRecord sorts correctly, I am trying to test that I have correctly coded the scope. I don't know how I would assert that the named scope is passing the correct sorting params. Can you enlighten me?
I would probably do something like this in RSpec:
require 'spec_helper' describe ClassWithNamedScope do describe 'scope :foo' do it 'exists' do ClassWithNamedScope.scopes.should be_include(:foo) end
it 'specifies that foo is true' do ClassWithNamedScope.stub(:named_scope)
ClassWithNamedScope.should_receive(:named_scope).with(:active, :conditions => {:foo => true}) load "#{RAILS_ROOT}/app/models/class_with_named_scope.rb" end end end
Similar stubbing and reloading should also work in test_unit with Mocha or somesuch, right?
> I would argue that it's unimportant -- even at the unit-level -- *how* > the test is passing. If every time you run the test it passes, then > the code does what you specify it does. When there's a problem, > identify the bug and write a new unit, integration, or functional test > that eliminates the unexpected behavior, then make that test pass.
I think effectively what you are saying is that it doesn't matter if it actually tests what it is supposed to as long as it passes. If you later discover that the app is not doing what it is supposed to then add more tests.
A good point. I would summarize it more like this: "If you verify, to the best of your ability, that the software behaves the way you want it to, then you're finished testing until you discover an exception. When you have a repeatable case where the software does not behave the way you expect, at that point you will have steps to cause the unexpected behavior (and therefore a test)."
I certainly won't argue with that.
It's not *only* the case that the test needs to pass; as you mentioned, it is also the case that the test needs to verify behavior. Sounds like we definitely agree on that. And you're right; it's that "to the best of your ability" that's tricky. Seems like that's where we went down different paths.
Agreed
> ...it's up to ActiveRecord to > hold up its end of the bargain.
I am not trying to test that ActiveRecord sorts correctly, I am trying to test that I have correctly coded the scope. I don't know how I would assert that the named scope is passing the correct sorting params. Can you enlighten me?
I would probably do something like this in RSpec:
require 'spec_helper' describe ClassWithNamedScope do describe 'scope :foo' do it 'exists' do ClassWithNamedScope.scopes.should be_include(:foo) end
it 'specifies that foo is true' do ClassWithNamedScope.stub(:named_scope)
ClassWithNamedScope.should_receive(:named_scope).with(:active, :conditions => {:foo => true}) load "#{RAILS_ROOT}/app/models/class_with_named_scope.rb" end end end
Similar stubbing and reloading should also work in test_unit with Mocha or somesuch, right?
Thanks for that, I see the principle. As I have said in other posts I think in practice provided I create the records in an arbitrary order that just checking that the test fails without the order clause in the named scope, and then passes with it, that the probability of this not being good enough is trivial.
Thanks again.
Colin
Colin Law wrote in post #965883: