using test data generators -- valid / invalid data

I'm investigating moving from fixtures to generated data (using Machinist right now, though this question is probably generic to any generator) but I seem to be missing one essential point:

If I use a fixture, I can test that e.g.

- an object will be valid with a valid owner (user) id - an object can't be valid with a nil owner id - an object can't be valid with a non-existent/invalid owner id

because I know what owner ids exist (and thus which don't exist).

If I use a generator to create test objects and owners, I have no idea what owner ids have been created, so how can I specify an invalid one for test purposes?

Or am I completely off-track in some way? TIA!

Hassan Schroeder wrote:

I'm investigating moving from fixtures to generated data (using Machinist right now, though this question is probably generic to any generator)

Good for you! I believe that fixtures are so poorly designed that they should be removed from the Rails core.

but I seem to be missing one essential point:

If I use a fixture, I can test that e.g.

- an object will be valid with a valid owner (user) id - an object can't be valid with a nil owner id - an object can't be valid with a non-existent/invalid owner id

because I know what owner ids exist (and thus which don't exist).

If I use a generator to create test objects and owners, I have no idea what owner ids have been created, so how can I specify an invalid one for test purposes?

You probably don't need to. But a couple of ideas come to mind: * User.max(:id) + 10 * Create a User, store the ID, delete the User. Voilà, invalid ID.

Or am I completely off-track in some way? TIA!

I think you are. validates_associated is silly and circularity-prone. You should be using foreign key constraints in the DB for this (the foreign_key_migrations and Foreigner plugins make this easy). And you don't need to test that in your app.

-- Hassan Schroeder ------------------------ hassan.schroeder@gmail.com twitter: @hassan

--

You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.

Best,

If I use a generator to create test objects and owners, I have no idea what owner ids have been created, so how can I specify an invalid one for test purposes?

You probably don't need to. But a couple of ideas come to mind: * User.max(:id) + 10

That didn't work but got me thinking in a workable direction, thanks.

... validates_associated is silly and circularity-prone. You should be using foreign key constraints in the DB for this

Well, I'll take that under advisement :slight_smile: but I would prefer to have the business logic in the app, and tested. Even if (as in this case) it's not actually TDD/BDD because I'm adding tests to someone else's code, the test suite now reflects intent -- better than the usual (which is to say no) documentation thereof.

Thanks,

Hassan Schroeder wrote:

If I use a generator to create test objects and owners, I have no idea what owner ids have been created, so how can I specify an invalid one for test purposes?

You probably don't need to. �But a couple of ideas come to mind: * User.max(:id) + 10

That didn't work but got me thinking in a workable direction, thanks.

Why didn't it work? That should be foolproof, unless I screwed up the syntax.

... validates_associated is silly and circularity-prone. You should be using foreign key constraints in the DB for this

Well, I'll take that under advisement :slight_smile: but I would prefer to have the business logic in the app, and tested.

Data integrity checking isn't business logic in the usual sense -- and it really is the job of the database.

Even if (as in this case) it's not actually TDD/BDD because I'm adding tests to someone else's code, the test suite now reflects intent -- better than the usual (which is to say no) documentation thereof.

Then have the validations as a backup, but remember that they're only a backup. Integrity constraints in the DB are not optional for a well-designed app.

And I repeat my suggestion to get rid of validates_associated. It makes extra queries and introduces possible circularities to do something that the DB already does better. It's reinventing a square wheel in the wrong place. If you need documentation, add a comment -- but I tend to think that the belongs_to statement will reveal enough intention.

You're falling into the common Rails trap of not respecting the database's intelligence. It's not dumb, and it should not be treated as if it were.

Thanks, -- Hassan Schroeder ------------------------ hassan.schroeder@gmail.com twitter: @hassan

Best,

Marnen Laibow-Koser wrote:

... validates_associated is silly and circularity-prone. You should be using foreign key constraints in the DB for this

Well, I'll take that under advisement :slight_smile: but I would prefer to have the business logic in the app, and tested.

Data integrity checking isn't business logic in the usual sense -- and it really is the job of the database.

+1

I strongly agree with Marnen on this point. Data integrity (especially referential integrity) is the database's responsibility. As far as I'm concerned business logic should add value to your model. Adding integrity at the database level is safer, smarter and allows you to save unnecessary test code in your application. Any reasonable database should contain it's own test code for verifying it knows how to validate data integrity.

You're falling into the common Rails trap of not respecting the database's intelligence. It's not dumb, and it should not be treated as if it were.

I have to say I don't always follow my own advice on this point. I catch myself not adding all the integrity constraints I really should have. but whenever the data is "important" there's no better way to ensure integrity than to do so in the database layer.

That didn't work but got me thinking in a workable direction, thanks.

Why didn't it work? That should be foolproof, unless I screwed up the syntax.

User.max(:id) + 10

NoMethodError: undefined method `max' for #<Class:0x101e8f060>

Data integrity checking isn't business logic in the usual sense -- and it really is the job of the database.

I'll disagree. In this case, requiring a valid user to be associated with an object *is* a business decision; tomorrow, I could decide to allow anonymous object creation, or anonymous creation in some specific circumstances only.

Committing a code fix to that effect is a lot better to my thinking than requiring everyone involved to make the comparable changes in all their DB instances (if it's even possible to handle that logic in the DB).

YMMV,

Hassan Schroeder wrote:

I'll disagree. In this case, requiring a valid user to be associated with an object *is* a business decision; tomorrow, I could decide to allow anonymous object creation, or anonymous creation in some specific circumstances only.

You just made my point quite nicely. In your case the association IS adding value to your model. I was referring to referential integrity that does not add business value to the application. As such it does not mandate test code at the application level, but integrity at the database level can still be important.

Hassan Schroeder wrote:

That didn't work but got me thinking in a workable direction, thanks.

Why didn't it work? �That should be foolproof, unless I screwed up the syntax.

User.max(:id) + 10

NoMethodError: undefined method `max' for #<Class:0x101e8f060>

OK, syntax goof. Sorry.

Data integrity checking isn't business logic in the usual sense -- and it really is the job of the database.

I'll disagree. In this case, requiring a valid user to be associated with an object *is* a business decision; tomorrow, I could decide to allow anonymous object creation, or anonymous creation in some specific circumstances only.

Then you can change your schema at that point. Till then, YAGNI.

Committing a code fix to that effect is a lot better to my thinking than requiring everyone involved to make the comparable changes in all their DB instances (if it's even possible to handle that logic in the DB).

WTF? Don't you use migrations? That's what they're for.

Anyway, crippling your current app because of a future situation that may not ever occur is extremely bad practice. Develop for what you have *now*.

Sorry, Hassan. Normally I respect your posts here very much, but in this case I think you're coming up with increasingly tenuous justifications for a very bad design decision.

YMMV, -- Hassan Schroeder ------------------------ hassan.schroeder@gmail.com twitter: @hassan

--

You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.

Best,

I may be completely out of line here… but

“WTF? Don’t you use migrations? That’s what they’re for.”

made me think that migrations aren’t the be all and end all. There are ALOT of things you cant do with migrations when it comes to database specific deployment. eg. triggers, user defined functions, even views… and even things like index creation is limited when using Postgres and MySQL or even SQLServer.

I know and realize Rails is opinionated software… but IMHO the community could benefit from using the power these dbms solutions offer. How about a DSL type plugin - not only enhancing migrations, but extending activerecord to embrace the DBMS rather than treating it as “just a storage mechanism”.

Sorry for hijacking here, just my two cents…

Uh, I'm not trying to "justify" anything. :slight_smile:

Perhaps what's been lost here is the original question -- how do you use test-data generators like Machinist/Sham when you need an example of *invalid* data, that is, values not in the generated set? It's simple with fixtures, eh, but...

Not every failing (TDD/BDD) test will have a solution implemented solely based on adding foreign keys to a DB schema.

Or so I imagine. :slight_smile:

Hassan Schroeder wrote:

Sorry, Hassan. �Normally I respect your posts here very much, but in this case I think you're coming up with increasingly tenuous justifications for a very bad design decision.

Uh, I'm not trying to "justify" anything. :slight_smile:

Perhaps I should have said "explain". Regardless, my criticisms are still as valid as they were.

Perhaps what's been lost here is the original question -- how do you use test-data generators like Machinist/Sham when you need an example of *invalid* data, that is, values not in the generated set? It's simple with fixtures, eh, but...

Only because you're putting the checking in the wrong place. And anyway, that's a red herring: i explained how to do something exactly equivalent with Machinist (though I think it's a bad idea).

Not every failing (TDD/BDD) test will have a solution implemented solely based on adding foreign keys to a DB schema.

Or so I imagine. :slight_smile:

Of course that's true. But foreign key constraints are the proper solution in your case -- so use them!

-- Hassan Schroeder ------------------------ hassan.schroeder@gmail.com twitter: @hassan

--

You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.

Best,

Johan De Klerk wrote:

I may be completely out of line here... but

"WTF? Don't you use migrations? That's what they're for."

made me think that migrations aren't the be all and end all.

How did what I wrote lead you to that conclusion?

There are ALOT of things you cant do with migrations when it comes to database specific deployment. eg. triggers, user defined functions, even views...

Migrations can do all of these things. The migration framework is extensible, as witness rails_sql_views and the various foreign key plugins, and as a last resort, you can put literal SQL in the migrations.

and even things like index creation is limited when using Postgres and MySQL or even SQLServer.

What about index creation doesn't Rails let you do?

I know and realize Rails is opinionated software... but IMHO the community could benefit from using the power these dbms solutions offer.

Yes. But I don't think putting large amounts of logic in the DB is a good idea. Key constraints are one thing; big stored procedures (such as I used to write) are quite another.

How about a DSL type plugin - not only enhancing migrations, but extending activerecord to embrace the DBMS rather than treating it as "just a storage mechanism".

Check out Hobo's "rich types". And I think there's a case to be made that ActiveRecord *does* embrace the DB; it's just that a lot of Rails developers don't.

Sorry for hijacking here, just my two cents...

Best,

Marnen, thanks for the feedback!

Perhaps I should do more googling on the subject… and I may be ignorant here but,

In terms of some of your comments:

Indexing is VERY db (and db version) specific. You may have situations where a btree index is more appropriate rather than a bitmap index. With Mysql (correct me if i’m wrong) there is no support for bitmap indexing. With Postgres there is… And with MySQL it very much depends on which storage engine your using.

Rails uses InnoDb by default? In some cases I may actually like to use MyISAM, which excels at super fast select queries, ideal for data warehousing situations.

I agree that putting large amounts of business logic in the DB is not a good idea. I’ve never liked using, creating or maintaining insanely complex stored procs… (thank you SQLServer 2000) IMO, most of the business logic should be in the application code - but then again - in situations where you just need to, for example, capture the state of a record before a update/delete for an audit trail, no rails code can compete with a simple trigger… coding that in Rails (similar to acts_as_versioned or is_paranoid) just seems counter intuitive to me.

Enough hijacking… I’ll check out hobos, thanks! And maybe start a new thread to more concisely state my thoughts…

Regards, Johan De Klerk

Sorry, you've completely missed the point.

The "use case" example I provided is utterly irrelevant. The data I want to test against could be a string, an IP address, anything.

I'm asking a generic question about using generators, so if there isn't an answer to my re-phrased question above, that's fine, and I'll continue using fixtures. (The Machinist list hasn't seemed to be able to come up with anything useful either.)

Hassan Schroeder wrote:

Perhaps what's been lost here is the original question -- how do you use test-data generators like Machinist/Sham when you need an example of *invalid* data, that is, values not in the generated set? It's simple with fixtures, eh, but...

Only because you're putting the checking in the wrong place. �And anyway, that's a red herring: i explained how to do something exactly equivalent with Machinist (though I think it's a bad idea).

Sorry, you've completely missed the point.

The "use case" example I provided is utterly irrelevant. The data I want to test against could be a string, an IP address, anything.

I'm asking a generic question about using generators,

No, I think you're the one who's missed the point. I gave you an easily genericized way of generating invalid factory data -- figure out the range of valid data, then generate something out-of-band. Simple.

I should say, though, that I rarely need to do this.

so if there isn't an answer to my re-phrased question above, that's fine, and I'll continue using fixtures. (The Machinist list hasn't seemed to be able to come up with anything useful either.)

Of course there's an answer. What part of my solution wasn't suitable?

-- Hassan Schroeder ------------------------ hassan.schroeder@gmail.com twitter: @hassan

--

You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.

Best,