Diff-testing? (Acceptance testing for laziness and legacy-ness)

I’ve come across a pattern where I’m working on some system without formal acceptance testing, but it’s known to be good in its current state.

My favorite way to quickly get to a place where I don’t regress is to spider the existing system, then save its output in text form, in the repository (usually in a directory called greenbar/).

This is useful for web systems, capturing the HTML, but it’s also useful for any system that at some point passes through a flattened-to-text output.

I plan on either using or writing a more generalized system. Do you know of one, or should I go ahead and write it?

An example of a project that does this is here: https://github.com/exad/zu/blob/master/test/run

With almost no intervention the tests end up producing a pretty thorough coverage.

Of course, this does not replace more detailed acceptance testing, and certainly does not replace unit testing, but it can be a useful tool.


A decent design would be:

  • A gem that helps you manage the spider/diff/commit loop
  • Gems that depend on that one to:
    • Use Capybara to help you spider web stuff better
    • Loop over invocations of CLI tools
    • Record various environments (that is, if you’re testing a tool that’s system-dependent, it would be good to be able to record the last time the run was snapshotted on various systems)

I have a few more ideas about it.

If you know of anything like this, or if you want to pair program on the creation of it, let me know.