Unary and Polyadic predicates, notin and notmatch predicates for Arel

Hi all. I was hoping to get some feedback on a patch to rails/arel I've put together. Not really sure where else to get some eyes on this, since Lighthouse doesn't seem appropriate given the separation of rails/arel from rails/rails.

http://gist.github.com/360075

This patch adds support for some new types of predicates and cleans up a few small things. Tests are included.

It adds a unary predicate, Not. Predicate#not will negate the predicate on which it's called.

For example, users[:name].eq('Bob').or(users[:name].eq('Joe')).not will match users not named Joe or Bob.

Polyadic predicates are _any and _all variations on the other operations, allowing multiple right-hand operands.

For example, users[:name].matches_any(['%Joe%', '%Bob%']) will match users with names containing Joe or Bob.

There is also a cleaner implementation of support for ranges with excluded end, and a refactor to the way that predication methods are defined. While the con to using metaprogramming for this is that they would be harder to eventually document, the pro, as I see it, is that it causes someone to think twice before creating predications with one-off behavior, and instead encourages engine-specific behavior (as with the previous "range with excluded end" implementation) to be pushed to the engine instead. I tried my best to stick with the general coding style there already in Arel::ClassExtensions in doing so.

Anyway, if a few folks have some time to take a look, any feedback would be greatly appreciated.

Err, minor correction.

users[:name].matches_any('%Joe%', '%Bob%') is the syntax. I wrote the
patch note last night and wasn't quite awake I guess. :slight_smile:

"Ernie Miller" <ernie@metautonomo.us> wrote in message news:22DF1A00-5287-4265-B18D-B3831E7E3586@metautonomo.us...

Hi all. I was hoping to get some feedback on a patch to rails/arel I've put >together. Not really sure where else to get some eyes on this, since >Lighthouse doesn't seem appropriate given the separation of rails/arel from >rails/rails.

arel.lighthouseapp.com would be appropriate, but since that appears to be extremely low volume, for additional feedback posting here makes sense.

http://gist.github.com/360075

This patch adds support for some new types of predicates and cleans up a few >small things. Tests are included.

It adds a unary predicate, Not. Predicate#not will negate the predicate on >which it's called.

For example, users[:name].eq('Bob').or(users[:name].eq('Joe')).not will >match users not named Joe or Bob.

That breaks left to right readability pretty badly. I really would like a better solution for negating predicates and predicate chains, but I'm not sure what.

You probably should also have the README file patched to indicate that AND and OR are implemented

Also, I'm not sure how of :notmatches_any and :notmatches_all will react. Will the negation be applied before or after the combining?

Also if you have a _any and _all prefix, should you not also include a _none, too?

Also is ".not" in use out in the wild already? How will existing code react? Will it error out indicating that the new .not toes not take parameters?

That breaks left to right readability pretty badly. I really would like a
better solution for negating predicates and predicate chains, but I'm not
sure what.

I agree that I don't much like the way it just dangles there like
that, but I don't see any other way to accomplish what's desired. In
working with it, I find that it helps to imagine yourself back in the
90s for a moment.

"User name equals Bill or user name equals Ted... NOT!"

OK, maybe I'm just odd. Or maybe the method name needs an exclamation
point. :slight_smile: In all seriousness, though, it's not much different than
something like String#reverse.

You probably should also have the README file patched to indicate that AND
and OR are implemented

The AND and OR the README refers to have been implemented for some
time.

Also, I'm not sure how of :notmatches_any and :notmatches_all will react.
Will the negation be applied before or after the combining?

Negation is on the predicate. notmatches_any will give you a predicate
something like:

(users.name NOT LIKE '%Joe%' OR users.name NOT LIKE '%Bob%)

The additional NOT would negate the whole thing:

NOT (users.name NOT LIKE '%Joe%' OR users.name NOT LIKE '%Bob%)

Also if you have a _any and _all prefix, should you not also include a
_none, too?

I don't necessarily think so, though I could add it if others think
otherwise. _none is the same as _any and not combined, and in the SQL
engine I'd say that's exactly how it'd have to be implemented, too.

Also is ".not" in use out in the wild already? How will existing code react?
Will it error out indicating that the new .not toes not take parameters?

The previous .not was a method on Arel::Attribute, not Arel::Predicate
-- I just renamed it to noteq because of the potential for exactly
that kind of confusion and for internal consistency . I'm not sure
what else out there is currently using Arel besides ActiveRecord, and
a search of the current source shows no use of Attribute#not in there.
It also only hit master on March 12th

I agree that I don't much like the way it just dangles there like
that, but I don't see any other way to accomplish what's desired. In
working with it, I find that it helps to imagine yourself back in the
90s for a moment.

"User name equals Bill or user name equals Ted... NOT!"

OK, maybe I'm just odd. Or maybe the method name needs an exclamation
point. :slight_smile: In all seriousness, though, it's not much different than
something like String#reverse.

Actually, I take that back. I updated the patch (still at
http://gist.github.com/360075) and added experimental support for an
alternate not syntax under Ruby 1.9, thanks to BasicObject#!.

articles = Article.scoped
articles.where(!articles.table[:title].eq('Hello!')).to_sql

=> "SELECT \"articles\".* FROM \"articles\" WHERE (NOT
(\"articles\".\"title\" = 'Hello!'))"

articles.where(not(articles.table[:title].eq('Hello!'))).to_sql

=> "SELECT \"articles\".* FROM \"articles\" WHERE (NOT
(\"articles\".\"title\" = 'Hello!'))"

It still passes the same tests it did before, and it seems (in this
case) reasonable to override the boolean not operator like this, to me.

"Ernie Miller" <ernie@metautonomo.us> wrote in message news:46299a9a-e38b-46c8-9305-ea94dbbd3160@r1g2000yqb.googlegroups.com...

I agree that I don't much like the way it just dangles there like
that, but I don't see any other way to accomplish what's desired. In
working with it, I find that it helps to imagine yourself back in the
90s for a moment.

"User name equals Bill or user name equals Ted... NOT!"

OK, maybe I'm just odd. Or maybe the method name needs an exclamation
point. :slight_smile: In all seriousness, though, it's not much different than
something like String#reverse.

Actually, I take that back. I updated the patch (still at
http://gist.github.com/360075) and added experimental support for an
alternate not syntax under Ruby 1.9, thanks to BasicObject#!.

articles = Article.scoped
articles.where(!articles.table[:title].eq('Hello!')).to_sql

=> "SELECT \"articles\".* FROM \"articles\" WHERE (NOT
(\"articles\".\"title\" = 'Hello!'))"

articles.where(not(articles.table[:title].eq('Hello!'))).to_sql

=> "SELECT \"articles\".* FROM \"articles\" WHERE (NOT
(\"articles\".\"title\" = 'Hello!'))"

I'm not fundemantally opposed to overriding the ! operator, as long as the result is strictly a locigal negation of the original, although some people may have a problem with it breaking the double negation to coerce to true boolean. Since that is a hack anyway, I don't much care for the loss.

I would however suggest that any documentation prefer the format used in the second example, as a way of encouraging readability. When the first case is used, an extra set of partentheses may help readability since somebody not particular familar with ruby's operator precedence could potentially misread it.

The result would definitely be a complement to the original, which
seems to me in keeping with the concept of "not" to begin with. In
fact, over the weekend I modified the patch to handle complements more
cleanly. That is, each predicate has a complement method which is used
behind the scenes. This prevents successive negations from causing the
accumulation of "nots" and using unnecessary memory with Not
predicates containing Not predicates containing... you get the idea.

Repeated nots toggle between the original and negated meaning, much
like multiple nots will do with boolean true/false:

uby-head > articles[:title].eq('Hello').to_sql
=> "\"articles\".\"title\" = 'Hello'"
ruby-head > (!articles[:title].eq('Hello')).to_sql
=> "\"articles\".\"title\" != 'Hello'"
ruby-head > (!!articles[:title].eq('Hello')).to_sql
=> "\"articles\".\"title\" = 'Hello'"

I think this cleans up the generated SQL considerably, though I'm not
sure from a DB/query optimization standpoint which one might be more
efficient. I'd think that depending on the DB engine, they should end
up being optimized identically, anyway, but I've been down that road
before and sometimes queries get optimized in strange ways.

Consider negation of a matches_any query. After the complement
changes:

ruby-head > articles[:title].matches_any('Hello%', 'Hi%').to_sql
=> "(\"articles\".\"title\" LIKE 'Hello%' OR \"articles\".\"title\"
LIKE 'Hi%')"
ruby-head > (!articles[:title].matches_any('Hello%', 'Hi%')).to_sql
=> "(\"articles\".\"title\" NOT LIKE 'Hello%' AND \"articles\".\"title
\" NOT LIKE 'Hi%')"

Previously, it would have yielded:

NOT(\"articles\".\"title\" LIKE 'Hello%' OR \"articles\".\"title\"
LIKE 'Hi%')

They're logically equivalent, but I'm not certain if some database
engines would optimize one better than the other.

-Ernie Miller
http://metautonomo.us

Well I have no further comments and since nobody ele appears to have anything to say, I'd submit the patch to arel's lighthouse (arel.lighthouseapp.com), and see if that gets any more response.