How to test sort order in named (or default) scope

Hi
I have a named scope that sets a sort order. In my unit test I can
run the query on a set of records (using Factory or fixtures) and
check the sort order is correct. I believe though that a find without
an explicit sort order may return the records in any order, so how can
I be sure that it is not just accidental that they are in the correct
order?

Colin

I have a named scope that sets a sort order. In my unit test I can
run the query on a set of records (using Factory or fixtures) and
check the sort order is correct. I believe though that a find without
an explicit sort order may return the records in any order, so how can
I be sure that it is not just accidental that they are in the correct
order?

Colin:

I would argue that it's unimportant -- even at the unit-level -- *how*
the test is passing. If every time you run the test it passes, then
the code does what you specify it does. When there's a problem,
identify the bug and write a new unit, integration, or functional test
that eliminates the unexpected behavior, then make that test pass.

I might also argue that it's not up to you to test ActiveRecord. You
might assert that your named scope is passing the correct sorting
parameters to ActiveRecord; at that point, it's up to ActiveRecord to
hold up its end of the bargain.

What do you think of that?

Andrew M. Kasper

I'd add that something that may give you some comfort is to endeavour
to write a failing test first.

Fred

Colin Law wrote in post #965435:

Hi
I have a named scope that sets a sort order. In my unit test I can
run the query on a set of records (using Factory or fixtures) and
check the sort order is correct. I believe though that a find without
an explicit sort order may return the records in any order, so how can
I be sure that it is not just accidental that they are in the correct
order?

Try creating the factories in a weird order. If they still are returned
in the correct order, then your sorting is correct.

...or...

Write the test so that it checks that the appropriate :order clause is
set. This is probably too invasive and implementation-dependent for my
taste, though I might do it occasionally.

Colin

Best,

I have a named scope that sets a sort order. In my unit test I can
run the query on a set of records (using Factory or fixtures) and
check the sort order is correct. I believe though that a find without
an explicit sort order may return the records in any order, so how can
I be sure that it is not just accidental that they are in the correct
order?

I would argue that it's unimportant -- even at the unit-level -- *how*
the test is passing. If every time you run the test it passes, then
the code does what you specify it does. When there's a problem,
identify the bug and write a new unit, integration, or functional test
that eliminates the unexpected behavior, then make that test pass.

I don't think that is a valid argument. I think effectively what you
are saying is that it doesn't matter if it actually tests what it is
supposed to as long as it passes. If you later discover that the app
is not doing what it is supposed to then add more tests.

I might also argue that it's not up to you to test ActiveRecord. You
might assert that your named scope is passing the correct sorting
parameters to ActiveRecord; at that point, it's up to ActiveRecord to
hold up its end of the bargain.

I am not trying to test that ActiveRecord sorts correctly, I am trying
to test that I have correctly coded the scope. I don't know how I
would assert that the named scope is passing the correct sorting
params. Can you enlighten me?

I think what I will do is use some common sense rather than being too
pedantic about the testing. By creating a number of records in a
random order and then checking that they are sorted correctly I think
the possibility of accidentally finding them come back in sorted order
is negligible. In addition I will have checked that the test fails
before I add the sort clause.

Colin

Absolutely, I will check that the test fails before adding the order
clause, but see my reply to Marnen below.

Colin

Colin Law wrote in post #965435:

Hi
I have a named scope that sets a sort order. In my unit test I can
run the query on a set of records (using Factory or fixtures) and
check the sort order is correct. I believe though that a find without
an explicit sort order may return the records in any order, so how can
I be sure that it is not just accidental that they are in the correct
order?

Try creating the factories in a weird order. If they still are returned
in the correct order, then your sorting is correct.

Probably. I may be wrong but I believe that postgreSQL does not even
guarantee that an unordered query will produce the same results if the
same query is run twice. Conceivably therefore I could test without
the order clause and check it fails, then, thinking I had added the
order clause re-run the test and find it pass. I think though as I
said in an earlier post that I can use common sense and assume the
probability of this is so small as to be insignificant.

...or...

Write the test so that it checks that the appropriate :order clause is
set. This is probably too invasive and implementation-dependent for my
taste, though I might do it occasionally.

I agree this seems like overkill and would likely be fragile.

Colin

Colin Law wrote in post #965538:

in the correct order, then your sorting is correct.

Probably. I may be wrong but I believe that postgreSQL does not even
guarantee that an unordered query will produce the same results if the
same query is run twice.

Right, SQL databases cannot be assumed to guarantee this. That's why I
suggested creating the factories in an arbitrary order, different from
what you want to see.

Conceivably therefore I could test without
the order clause and check it fails, then, thinking I had added the
order clause re-run the test and find it pass. I think though as I
said in an earlier post that I can use common sense and assume the
probability of this is so small as to be insignificant.

Well, if you create the factories in order, then there's a significant
probability of this happening...

Best,

Colin Law wrote in post #965538:

in the correct order, then your sorting is correct.

Probably. I may be wrong but I believe that postgreSQL does not even
guarantee that an unordered query will produce the same results if the
same query is run twice.

Right, SQL databases cannot be assumed to guarantee this. That's why I
suggested creating the factories in an arbitrary order, different from
what you want to see.

I don't think I made my point clearly. I believe that, in theory,
even by creating the objects in an arbitrary order one cannot
guarantee that an un-ordered query will not coincidentally end up with
an ordered set. I do also believe however that the probability of
this is so small as to considered negligible. It is still an
interesting academic point if not an issue in practice.

Conceivably therefore I could test without
the order clause and check it fails, then, thinking I had added the
order clause re-run the test and find it pass. I think though as I
said in an earlier post that I can use common sense and assume the
probability of this is so small as to be insignificant.

Well, if you create the factories in order, then there's a significant
probability of this happening...

Certainly.

Colin

Not that I think that this is the best way, actually, thats probably
the worst, but on the same time, it is the easier:

/ORDER BY column/ =~ Model.your_scope.to_sql

I had not thought of doing something like that. Thanks

Colin

> I would argue that it's unimportant -- even at the unit-level -- *how*
> the test is passing. If every time you run the test it passes, then
> the code does what you specify it does. When there's a problem,
> identify the bug and write a new unit, integration, or functional test
> that eliminates the unexpected behavior, then make that test pass.

I think effectively what you
are saying is that it doesn't matter if it actually tests what it is
supposed to as long as it passes. If you later discover that the app
is not doing what it is supposed to then add more tests.

A good point. I would summarize it more like this:
"If you verify, to the best of your ability, that the software behaves
the way you want it to, then you're finished testing until you
discover an exception. When you have a repeatable case where the
software does not behave the way you expect, at that point you will
have steps to cause the unexpected behavior (and therefore a test)."

It's not *only* the case that the test needs to pass; as you
mentioned, it is also the case that the test needs to verify behavior.
Sounds like we definitely agree on that. And you're right; it's that
"to the best of your ability" that's tricky. Seems like that's where
we went down different paths.

> ...it's up to ActiveRecord to
> hold up its end of the bargain.

I am not trying to test that ActiveRecord sorts correctly, I am trying
to test that I have correctly coded the scope. I don't know how I
would assert that the named scope is passing the correct sorting
params. Can you enlighten me?

I would probably do something like this in RSpec:

require 'spec_helper'
describe ClassWithNamedScope do
  describe 'scope :foo' do
    it 'exists' do
      ClassWithNamedScope.scopes.should be_include(:foo)
    end

    it 'specifies that foo is true' do
      ClassWithNamedScope.stub(:named_scope)

ClassWithNamedScope.should_receive(:named_scope).with(:active, :conditions
=> {:foo => true})
      load "#{RAILS_ROOT}/app/models/class_with_named_scope.rb"
    end
  end
end

Similar stubbing and reloading should also work in test_unit with
Mocha or somesuch, right?

> I would argue that it's unimportant -- even at the unit-level -- *how*
> the test is passing. If every time you run the test it passes, then
> the code does what you specify it does. When there's a problem,
> identify the bug and write a new unit, integration, or functional test
> that eliminates the unexpected behavior, then make that test pass.

I think effectively what you
are saying is that it doesn't matter if it actually tests what it is
supposed to as long as it passes. If you later discover that the app
is not doing what it is supposed to then add more tests.

A good point. I would summarize it more like this:
"If you verify, to the best of your ability, that the software behaves
the way you want it to, then you're finished testing until you
discover an exception. When you have a repeatable case where the
software does not behave the way you expect, at that point you will
have steps to cause the unexpected behavior (and therefore a test)."

I certainly won't argue with that.

It's not *only* the case that the test needs to pass; as you
mentioned, it is also the case that the test needs to verify behavior.
Sounds like we definitely agree on that. And you're right; it's that
"to the best of your ability" that's tricky. Seems like that's where
we went down different paths.

Agreed

> ...it's up to ActiveRecord to
> hold up its end of the bargain.

I am not trying to test that ActiveRecord sorts correctly, I am trying
to test that I have correctly coded the scope. I don't know how I
would assert that the named scope is passing the correct sorting
params. Can you enlighten me?

I would probably do something like this in RSpec:

require 'spec_helper'
describe ClassWithNamedScope do
describe 'scope :foo' do
it 'exists' do
ClassWithNamedScope.scopes.should be_include(:foo)
end

it 'specifies that foo is true' do
ClassWithNamedScope.stub(:named_scope)

ClassWithNamedScope.should_receive(:named_scope).with(:active, :conditions
=> {:foo => true})
load "#{RAILS_ROOT}/app/models/class_with_named_scope.rb"
end
end
end

Similar stubbing and reloading should also work in test_unit with
Mocha or somesuch, right?

Thanks for that, I see the principle. As I have said in other posts I
think in practice provided I create the records in an arbitrary order
that just checking that the test fails without the order clause in the
named scope, and then passes with it, that the probability of this not
being good enough is trivial.

Thanks again.

Colin

Colin Law wrote in post #965883: