using test data generators -- valid / invalid data

I'm investigating moving from fixtures to generated data (using
Machinist right now, though this question is probably generic to
any generator) but I seem to be missing one essential point:

If I use a fixture, I can test that e.g.

- an object will be valid with a valid owner (user) id
- an object can't be valid with a nil owner id
- an object can't be valid with a non-existent/invalid owner id

because I know what owner ids exist (and thus which don't exist).

If I use a generator to create test objects and owners, I have no
idea what owner ids have been created, so how can I specify an
invalid one for test purposes?

Or am I completely off-track in some way? TIA!

Hassan Schroeder wrote:

I'm investigating moving from fixtures to generated data (using
Machinist right now, though this question is probably generic to
any generator)

Good for you! I believe that fixtures are so poorly designed that they
should be removed from the Rails core.

but I seem to be missing one essential point:

If I use a fixture, I can test that e.g.

- an object will be valid with a valid owner (user) id
- an object can't be valid with a nil owner id
- an object can't be valid with a non-existent/invalid owner id

because I know what owner ids exist (and thus which don't exist).

If I use a generator to create test objects and owners, I have no
idea what owner ids have been created, so how can I specify an
invalid one for test purposes?

You probably don't need to. But a couple of ideas come to mind:
* User.max(:id) + 10
* Create a User, store the ID, delete the User. Voilà, invalid ID.

Or am I completely off-track in some way? TIA!

I think you are. validates_associated is silly and circularity-prone.
You should be using foreign key constraints in the DB for this (the
foreign_key_migrations and Foreigner plugins make this easy). And you
don't need to test that in your app.

--
Hassan Schroeder ------------------------ hassan.schroeder@gmail.com
twitter: @hassan

--

You received this message because you are subscribed to the Google
Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Best,

If I use a generator to create test objects and owners, I have no
idea what owner ids have been created, so how can I specify an
invalid one for test purposes?

You probably don't need to. But a couple of ideas come to mind:
* User.max(:id) + 10

That didn't work but got me thinking in a workable direction, thanks.

... validates_associated is silly and circularity-prone.
You should be using foreign key constraints in the DB for this

Well, I'll take that under advisement :slight_smile: but I would prefer to have the
business logic in the app, and tested. Even if (as in this case) it's not
actually TDD/BDD because I'm adding tests to someone else's code,
the test suite now reflects intent -- better than the usual (which is to
say no) documentation thereof.

Thanks,

Hassan Schroeder wrote:

If I use a generator to create test objects and owners, I have no
idea what owner ids have been created, so how can I specify an
invalid one for test purposes?

You probably don't need to. �But a couple of ideas come to mind:
* User.max(:id) + 10

That didn't work but got me thinking in a workable direction, thanks.

Why didn't it work? That should be foolproof, unless I screwed up the
syntax.

... validates_associated is silly and circularity-prone.
You should be using foreign key constraints in the DB for this

Well, I'll take that under advisement :slight_smile: but I would prefer to have the
business logic in the app, and tested.

Data integrity checking isn't business logic in the usual sense -- and
it really is the job of the database.

Even if (as in this case) it's
not
actually TDD/BDD because I'm adding tests to someone else's code,
the test suite now reflects intent -- better than the usual (which is to
say no) documentation thereof.

Then have the validations as a backup, but remember that they're only a
backup. Integrity constraints in the DB are not optional for a
well-designed app.

And I repeat my suggestion to get rid of validates_associated. It makes
extra queries and introduces possible circularities to do something that
the DB already does better. It's reinventing a square wheel in the
wrong place. If you need documentation, add a comment -- but I tend to
think that the belongs_to statement will reveal enough intention.

You're falling into the common Rails trap of not respecting the
database's intelligence. It's not dumb, and it should not be treated as
if it were.

Thanks,
--
Hassan Schroeder ------------------------ hassan.schroeder@gmail.com
twitter: @hassan

Best,

Marnen Laibow-Koser wrote:

... validates_associated is silly and circularity-prone.
You should be using foreign key constraints in the DB for this

Well, I'll take that under advisement :slight_smile: but I would prefer to have the
business logic in the app, and tested.

Data integrity checking isn't business logic in the usual sense -- and
it really is the job of the database.

+1

I strongly agree with Marnen on this point. Data integrity (especially
referential integrity) is the database's responsibility. As far as I'm
concerned business logic should add value to your model. Adding
integrity at the database level is safer, smarter and allows you to save
unnecessary test code in your application. Any reasonable database
should contain it's own test code for verifying it knows how to validate
data integrity.

You're falling into the common Rails trap of not respecting the
database's intelligence. It's not dumb, and it should not be treated as
if it were.

I have to say I don't always follow my own advice on this point. I catch
myself not adding all the integrity constraints I really should have.
but whenever the data is "important" there's no better way to ensure
integrity than to do so in the database layer.

That didn't work but got me thinking in a workable direction, thanks.

Why didn't it work? That should be foolproof, unless I screwed up the
syntax.

User.max(:id) + 10

NoMethodError: undefined method `max' for #<Class:0x101e8f060>

Data integrity checking isn't business logic in the usual sense -- and
it really is the job of the database.

I'll disagree. In this case, requiring a valid user to be associated with
an object *is* a business decision; tomorrow, I could decide to allow
anonymous object creation, or anonymous creation in some specific
circumstances only.

Committing a code fix to that effect is a lot better to my thinking than
requiring everyone involved to make the comparable changes in all
their DB instances (if it's even possible to handle that logic in the DB).

YMMV,

Hassan Schroeder wrote:

I'll disagree. In this case, requiring a valid user to be associated
with
an object *is* a business decision; tomorrow, I could decide to allow
anonymous object creation, or anonymous creation in some specific
circumstances only.

You just made my point quite nicely. In your case the association IS
adding value to your model. I was referring to referential integrity
that does not add business value to the application. As such it does not
mandate test code at the application level, but integrity at the
database level can still be important.

Hassan Schroeder wrote:

That didn't work but got me thinking in a workable direction, thanks.

Why didn't it work? �That should be foolproof, unless I screwed up the
syntax.

User.max(:id) + 10

NoMethodError: undefined method `max' for #<Class:0x101e8f060>

OK, syntax goof. Sorry.

Data integrity checking isn't business logic in the usual sense -- and
it really is the job of the database.

I'll disagree. In this case, requiring a valid user to be associated
with
an object *is* a business decision; tomorrow, I could decide to allow
anonymous object creation, or anonymous creation in some specific
circumstances only.

Then you can change your schema at that point. Till then, YAGNI.

Committing a code fix to that effect is a lot better to my thinking than
requiring everyone involved to make the comparable changes in all
their DB instances (if it's even possible to handle that logic in the
DB).

WTF? Don't you use migrations? That's what they're for.

Anyway, crippling your current app because of a future situation that
may not ever occur is extremely bad practice. Develop for what you have
*now*.

Sorry, Hassan. Normally I respect your posts here very much, but in
this case I think you're coming up with increasingly tenuous
justifications for a very bad design decision.

YMMV,
--
Hassan Schroeder ------------------------ hassan.schroeder@gmail.com
twitter: @hassan

--

You received this message because you are subscribed to the Google
Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Best,

I may be completely out of line here… but

“WTF? Don’t you use migrations? That’s what they’re for.”

made me think that migrations aren’t the be all and end all. There are ALOT of things you cant do with migrations when it comes to database specific deployment. eg. triggers, user defined functions, even views… and even things like index creation is limited when using Postgres and MySQL or even SQLServer.

I know and realize Rails is opinionated software… but IMHO the community could benefit from using the power these dbms solutions offer. How about a DSL type plugin - not only enhancing migrations, but extending activerecord to embrace the DBMS rather than treating it as “just a storage mechanism”.

Sorry for hijacking here, just my two cents…

Uh, I'm not trying to "justify" anything. :slight_smile:

Perhaps what's been lost here is the original question -- how do you
use test-data generators like Machinist/Sham when you need an
example of *invalid* data, that is, values not in the generated set?
It's simple with fixtures, eh, but...

Not every failing (TDD/BDD) test will have a solution implemented
solely based on adding foreign keys to a DB schema.

Or so I imagine. :slight_smile:

Hassan Schroeder wrote:

Sorry, Hassan. �Normally I respect your posts here very much, but in
this case I think you're coming up with increasingly tenuous
justifications for a very bad design decision.

Uh, I'm not trying to "justify" anything. :slight_smile:

Perhaps I should have said "explain". Regardless, my criticisms are
still as valid as they were.

Perhaps what's been lost here is the original question -- how do you
use test-data generators like Machinist/Sham when you need an
example of *invalid* data, that is, values not in the generated set?
It's simple with fixtures, eh, but...

Only because you're putting the checking in the wrong place. And
anyway, that's a red herring: i explained how to do something exactly
equivalent with Machinist (though I think it's a bad idea).

Not every failing (TDD/BDD) test will have a solution implemented
solely based on adding foreign keys to a DB schema.

Or so I imagine. :slight_smile:

Of course that's true. But foreign key constraints are the proper
solution in your case -- so use them!

--
Hassan Schroeder ------------------------ hassan.schroeder@gmail.com
twitter: @hassan

--

You received this message because you are subscribed to the Google
Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Best,

Johan De Klerk wrote:

I may be completely out of line here... but

"WTF? Don't you use migrations? That's what they're for."

made me think that migrations aren't the be all and end all.

How did what I wrote lead you to that conclusion?

There are
ALOT
of things you cant do with migrations when it comes to database specific
deployment. eg. triggers, user defined functions, even views...

Migrations can do all of these things. The migration framework is
extensible, as witness rails_sql_views and the various foreign key
plugins, and as a last resort, you can put literal SQL in the
migrations.

and even
things like index creation is limited when using Postgres and MySQL or
even
SQLServer.

What about index creation doesn't Rails let you do?

I know and realize Rails is opinionated software... but IMHO the
community
could benefit from using the power these dbms solutions offer.

Yes. But I don't think putting large amounts of logic in the DB is a
good idea. Key constraints are one thing; big stored procedures (such
as I used to write) are quite another.

How about
a
DSL type plugin - not only enhancing migrations, but extending
activerecord
to embrace the DBMS rather than treating it as "just a storage
mechanism".

Check out Hobo's "rich types". And I think there's a case to be made
that ActiveRecord *does* embrace the DB; it's just that a lot of Rails
developers don't.

Sorry for hijacking here, just my two cents...

Best,

Marnen, thanks for the feedback!

Perhaps I should do more googling on the subject… and I may be ignorant here but,

In terms of some of your comments:

Indexing
is VERY db (and db version) specific. You may have situations where a
btree index is more appropriate rather than a bitmap index. With Mysql
(correct me if i’m wrong) there is no support for bitmap indexing. With
Postgres there is… And with MySQL it very much depends on which
storage engine your using.

Rails uses InnoDb by default? In some
cases I may actually like to use MyISAM, which excels at super fast
select queries, ideal for data warehousing situations.

I agree
that putting large amounts of business logic in the DB is not a good
idea. I’ve never liked using, creating or maintaining insanely complex
stored procs… (thank you SQLServer 2000) IMO, most of the business
logic should be in the application code - but then again - in
situations where you just need to, for example, capture the state of a
record before a update/delete for an audit trail, no rails code can
compete with a simple trigger… coding that in Rails (similar to acts_as_versioned or is_paranoid) just seems counter
intuitive to me.

Enough hijacking… I’ll check out hobos, thanks! And maybe start a new thread to more concisely state my thoughts…

Regards,
Johan De Klerk

Sorry, you've completely missed the point.

The "use case" example I provided is utterly irrelevant. The data I
want to test against could be a string, an IP address, anything.

I'm asking a generic question about using generators, so if there
isn't an answer to my re-phrased question above, that's fine, and
I'll continue using fixtures. (The Machinist list hasn't seemed to be
able to come up with anything useful either.)

Hassan Schroeder wrote:

Perhaps what's been lost here is the original question -- how do you
use test-data generators like Machinist/Sham when you need an
example of *invalid* data, that is, values not in the generated set?
It's simple with fixtures, eh, but...

Only because you're putting the checking in the wrong place. �And
anyway, that's a red herring: i explained how to do something exactly
equivalent with Machinist (though I think it's a bad idea).

Sorry, you've completely missed the point.

The "use case" example I provided is utterly irrelevant. The data I
want to test against could be a string, an IP address, anything.

I'm asking a generic question about using generators,

No, I think you're the one who's missed the point. I gave you an easily
genericized way of generating invalid factory data -- figure out the
range of valid data, then generate something out-of-band. Simple.

I should say, though, that I rarely need to do this.

so if there
isn't an answer to my re-phrased question above, that's fine, and
I'll continue using fixtures. (The Machinist list hasn't seemed to be
able to come up with anything useful either.)

Of course there's an answer. What part of my solution wasn't suitable?

--
Hassan Schroeder ------------------------ hassan.schroeder@gmail.com
twitter: @hassan

--

You received this message because you are subscribed to the Google
Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.

Best,