Inconsistent test results

Hello All,
I seem to be having recurring problems in that my tests work in some
cases, and not in others.

My OS is Ubuntu 6.10 - the Edgy Eft and I'm using:

RadRails 0.7.1
Ruby 1.8.4
MySQL 5.0.24a-Debian_9-log
Rails 1.1.6

To give an example yesterday I had an issue where a particular unit
test was failing when run as part of a group of unit tests. However,
running this test alone passed.

Furthermore, today I have a situation where my an assert_tag in my
functional tests is failing from the command line

rake test:functionals

However running the functional tests from RadRails is working. Is
anyone else experiencing inconsistent behaviour like this? I find
myself spending most of the time trying to resolve these issues rather
than building functionality. This is causing me huge frustration so any
assistance would be greatly appreciated.

Regards,
John

JohnW wrote:

To give an example yesterday I had an issue where a particular unit
test was failing when run as part of a group of unit tests. However,
running this test alone passed.

Most likely you have inter suite dependency problems. You need to make
sure that all testcases have the needed fixtures instead of accidentally
reusing garbage. If using transactional fixtures, check by looking at
log/test.log that "BEGIN" "ROLLBACK" are called before the test code
itself. You can also disable transactional fixtures for selected suites.

Zsombor

Hi Zsombor,
I have transactions turned on in test_helper.

Can you tell me how to disable transactional fixtures for selected suites? I’d like to try that and see the results?

Thanks,
John

John Ward wrote:

Can you tell me how to disable transactional fixtures for selected
suites? I'd like to try that and see the results?

Basically you just set the use_transactional_fixtures class level variable
to false. For example to disable for one particular suite:

class SlowGroupsControllerTest < Test::Unit::TestCase
  fixtures :groups, :users

  # turn transactions off since we have atleast one transaction
  # dependent testcase. With mysql transactions cannot be nested.
  self.use_transactional_fixtures = false
  #.....
  # your tests here
  #.....
end

Best,
Zsombor

There is a known bug in Rails when the assert_tag is used in the
integration tests

Check out Mark Lunds website for more information:
http://wiki.marklunds.com/index.php?title=Test_Driven_Development_with_Ruby

To fix this replace the old html_document method in test_process.rb
with the following:

    def html_document
      HTML::Document.new(@response.body)
    end

Give it a shot and let us know if that solves your problem.

Hi All,
Thanks for your replies. My issues were because of a combination of problems, and really only identified by isolating the issues by taking the following actions.

  1. Turn off transactional fixtures
  2. Running tests individually using ruby test/…/my_test.rb --name test_my_method

When the test failed I was able to integrate the database and see its state, which helped me establish the core problem.

Another issue I had was with caching in my functional tests. For instance

def test_something

bill = users(:bill)

#suppose User has a has_one :email

#do some functional testing to remove email

#check bill
assert_nil bill.email

end

In the above the bill.email
has failing, though the related record had been deleted.

The following reloads the association and works

assert_nil bill.email(true)

Regards,
John