Robert Walker wrote:
Brian Mcardle wrote:
I'm still a relative newbie to RoR, but I'm finding it a very powerful
tool. We have a couple of RoR applications up and in production, and
we're making good use of the automatic testing to make sure they're bug
free.
What I would like to do is to test using the MySQL data our users have
entered. Either using that data to create fixtures, or copying the
production db into the test db on every test run. With the test db
recreated on every run, obviously a standard outside dump won't work.
This would be a far more comprehensive test than the few fixtures we've
manually created. Is there an easy way to do this?
Personally, I disagree with this assessment. If you're finding problems
with the data users are entering you've already failed them, and
possibly lost their business.
I believe that it is our responsibility, as developers, to consider all
possible conditions that could make something fail. Obviously this is an
impossible task. There will be some edge cases that get overlooked.
The reason I disagree with your proposed approach is that it could very
easily lead to developer complacency. Plan for zero failures by writing
examples to test every edge case you can imagine. Some will be missed,
however, I would deal with these on a case-by-case basis by ensuring you
back your testing with excellent failure notification.
Also, plan for a comprehensive beta testing phase. For any notifications
you receive, write an example to cover whatever was missed. If you
fulfilled your part of the "contract" as a developer, by creating
exhaustive tests, then these reports should be relatively rare by the
time you go into production. If you still have a lot of failures in
production, then you simply haven't done your job.
P.S. This response is very much an opinion, and you know what opinions
are often compared with. And, I really don't mean to be one.
Thanks for the thoughts. I certainly agree with the general thrust of
your statement - that there should be a thoroughly complete battery of
tests on an application (if I understand you correctly).
But as you point out, this is impossible. We'll say I never thought of
testing for non-English characters in certain strings. All my fixtures
are English, all my tests pass. One of my users creates a record with a
'á' character... maybe nothing breaks. Maybe on a later upgrade, I add a
new page, and that breaks.
If I had a test that used the real data, including the 'á', I would've
spotted this before deploying to production, and saved myself
embarrassment. As it is, every failure gets covered by new tests or
monitoring.
I don't imagine these errors (those caught by db data tests) to occur
more than once in a blue moon, but I believe testing with more than my
manual fixtures will help me create a application with less bugs, and
surely that's always a good thing?
PS I do sincerely thank you for your opinion. RoR represents a new way
of thinking, and of development for me. It's been a year, but some of it
is still a little ...alien to me.
