adventures with rails, cucumber, selenium, and a database

rubyonrails-talk
1

I have a rails 3.0 application with complicated logic and was finding that changes to fix a bug would introduce another bug elsewhere. I needed an automatic regression test tool so I could quickly know if this happened. I am using cucumber for this. I know that I am not doing BDD or TDD, but that is beside the point.

My initial set of scenarios was developed using capybara and seeding the database with test fixtures. Although it mostly worked, there were problems because it was not exercizing the javascript on my web page, so I switched to selenium. Now none of my scenarios worked. sqlite3 was complaining about the database being locked because it can only handle one request at a time. I tried switching to a mysql test database, but then the scenarios did not see the changes the application made to database. After much googling, I found that both of these problems were because selenium runs in a separate thread while capybara does not. The suggested solution for this was to change the database cleaner strategy from transaction to truncate. After this change, most of the scenarios ran, but for those using the scenario outline, only the first case would pass. The following cases all found an empty database. Truncate was deleting all the database records after the first case and not restoring it. After more googling I found I could set the database cleaner strategy to nil. Now all of my scenarios pass, but I have to be careful that no two scenarios use the same database records because database changes are not cleared between scenarios. I also have been able to go back to using sqlite3.

Is there a better alternative?

2

Without a single doubt, using factory_girl +database_cleaner gem. Are you using the test framework from rails? or Rspec?

here is a good episode #275 How I Test - RailsCasts explaining how to integrate this. and probably ehre talks about database cleaner https://gist.github.com/docwhat/1190475. try to look up yourself some more information.

your tests should be as isolated as possible, so your next test shouldnt depend on if the one before fails or passes and what does on the database.

3

I run the tests using:

$ bundle exec cucumber

It is my understanding that cucumber is built on top of rspec.

I am also using test/fixtures and not factory_girl. The reason is that I am rewriting a perl terminal interface tool as a web application. The legacy database tables go back 10 years. The data for any scenario is easily extracted from this database and converted into yaml. The typical scenario uses 50-100 rows from 10-30 tables. It would take forever to write all this data as factory_girl ruby code.

It is very annoying that I had to resort to using the nil option for the database cleaner, but as I explained, neither transaction nor truncate works.

4

So are you still using sqlite3 to run the tests? why didnt the tests see the difference on the database when using MySQL?

5

The issue with cucumber, selenium and database transactions is described in:

http://www.datatravels.com/technotes/2011/10/01/cucumber-capybara-with-selenium-and-truncated-fixt/

I also found suggested work-arounds at

http://pastie.org/1745020 http://blog.thefrontiergroup.com.au/2009/10/database-transactions-in-cucumber-breaking-selenium/

but these do not work for me.