Hello, all!
I’ve been a subscriber for a bit, but now I have a question for those more knowledgeable than I.
Background:
I’ve inherited a Rails 2.3.x app, and have had some issues installing the right versions of rspec / rubygems / etc. to find compatible versions, but I pretty much have discovered that I may need a modified approach to testing other than the straightforward tutorials. The app came with no working tests (although there are stubs for fixtures-based tests inheriting from ActionController::TestCase). I’ve been playing with Rails part-time but this is my first “big” project. We’ll be doing significant mods soon, but I don’t want to do too much until we have a good testing strategy. I replaced a manual deploy with capistrano, etc. so we’re pretty ok for small mods but wholesale changes scare me without any tests.
The application doesn’t manage its own login – the accounts / passwords are stored on an external system. There is a fairly intimate coordination of data between the external system and the rails app: authentication takes place remotely, but app permissions are stored on the rails side. The app will modify data both on the Rails side and the external side, but some initial setup on the external site is needed before anything can happen on Rails, and periodic tasks have to be performed on the external server (manually) in order to simulate a true workflow. So, we can’t really authenticate in a test, or do anything meaningful, without setup on both the external system and the rails system. This leads me to believe that using the ‘test’ database, which is regenerated / rolled back often, is a problem. I also can’t believe that I’m the only one who faces issues like this in designing tests for a Rails application.
Additional complications include the following:
-
Data must be uploaded to the rails side, processed, and then sent over manually to the external server, and processed there with multiple human interactions. Only after that happens can some of the tests be performed on the rails side, and the tests change both rails data and external data. If a test were to be performed after all that setup, and the rails side of the database was to be “rolled back”, there would be inconsistencies between the rails db and the external db which might confound any future test results without cleaning everything up and starting over again, something which would take between 10 and 50 minutes to create (depending upon how much of a “clean slate” we want on both sides).
-
We have dependencies upon the web server (webrick doesn’t work, and only older versions of passenger work). This is due to some nasty encoding we do to send encoded ‘/’ characters via URL params, and we end up with double encodings with certain servers. If any of the test mechanisms also have this issue with sending / retrieving / and generating links with the ‘/’ encoded URL parameters, we’d again have confounding results. I’d like to break this dependency, but again, it would be nice to have some tests first!
I’ve thought of possibly using an existing set of test data on the remote side (with matched data on the Rails side), and running the tests with the data on the rails side until we’ve “used up” that data set (after certain sets of operations on the rails side, the particular data item is no longer modifiable in the same manner), but that doesn’t appear to be obvious how I’d do that with either unit tests / fixtures or rspec. Additionally, uploading a file in a particular format is an integral step in many of the rails-side tests – integrating that into the testing framework would be nice, but there’s probably a lot that could be done before we had to do that.
It almost seems that we need to have a GUI-based driver for the more complicated things (such as uploading files / checking results). On the other hand, the basic get requests do require a lot of setup (as I’ve described above) that doesn’t seem obvious to me with the current test options.
Any suggestions for test infrastructure gems, tutorials, blog posts, etc. that might be of use would be appreciated. Right now, I’d rather get the tests working rather than try to bring this app up to 3.x in order to pull in the fanciest new tools 
If anyone has some ideas and needs more info, I’ll try to provide that as well, it seems like this post is long enough already!
Thanks and best wishes,
- Rholmes
