Inconsistent test results

rubyonrails-talk
1

Hello All, I seem to be having recurring problems in that my tests work in some cases, and not in others.

My OS is Ubuntu 6.10 - the Edgy Eft and I'm using:

RadRails 0.7.1 Ruby 1.8.4 MySQL 5.0.24a-Debian_9-log Rails 1.1.6

To give an example yesterday I had an issue where a particular unit test was failing when run as part of a group of unit tests. However, running this test alone passed.

Furthermore, today I have a situation where my an assert_tag in my functional tests is failing from the command line

rake test:functionals

However running the functional tests from RadRails is working. Is anyone else experiencing inconsistent behaviour like this? I find myself spending most of the time trying to resolve these issues rather than building functionality. This is causing me huge frustration so any assistance would be greatly appreciated.

Regards, John

2

JohnW wrote:

To give an example yesterday I had an issue where a particular unit test was failing when run as part of a group of unit tests. However, running this test alone passed.

Most likely you have inter suite dependency problems. You need to make sure that all testcases have the needed fixtures instead of accidentally reusing garbage. If using transactional fixtures, check by looking at log/test.log that "BEGIN" "ROLLBACK" are called before the test code itself. You can also disable transactional fixtures for selected suites.

Zsombor

3

Hi Zsombor, I have transactions turned on in test_helper.

Can you tell me how to disable transactional fixtures for selected suites? I’d like to try that and see the results?

Thanks, John

4

John Ward wrote:

Can you tell me how to disable transactional fixtures for selected suites? I'd like to try that and see the results?

Basically you just set the use_transactional_fixtures class level variable to false. For example to disable for one particular suite:

class SlowGroupsControllerTest < Test::Unit::TestCase   fixtures :groups, :users

  # turn transactions off since we have atleast one transaction   # dependent testcase. With mysql transactions cannot be nested.   self.use_transactional_fixtures = false   #.....   # your tests here   #..... end

Best, Zsombor

5

There is a known bug in Rails when the assert_tag is used in the integration tests

Check out Mark Lunds website for more information: http://wiki.marklunds.com/index.php?title=Test_Driven_Development_with_Ruby

To fix this replace the old html_document method in test_process.rb with the following:

    def html_document       HTML::Document.new(@response.body)     end

Give it a shot and let us know if that solves your problem.

6

Hi All, Thanks for your replies. My issues were because of a combination of problems, and really only identified by isolating the issues by taking the following actions.

  1. Turn off transactional fixtures
  2. Running tests individually using ruby test/…/my_test.rb --name test_my_method

When the test failed I was able to integrate the database and see its state, which helped me establish the core problem.

Another issue I had was with caching in my functional tests. For instance

def test_something

bill = users(:bill)

#suppose User has a has_one :email

#do some functional testing to remove email …

#check bill assert_nil bill.email

end

In the above the bill.email has failing, though the related record had been deleted.

The following reloads the association and works

assert_nil bill.email(true)

Regards, John