In-memory uniqueness validator

Hi,

Currently uniqueness validator uses database queries which not work for nested forms.

For example, I have a company form which have many partners and each partner have a person and a percentage.

Using uniqueness validator, I can not guarantee someone will not be a partner two times because the partners aren’t on database yet.

We’ve implemented a validator similar to this one: http://pastie.org/private/osa3pmono5l1ykrd7vipwa

I’m not sure if the uniqueness validator should handle it.

Take a look: http://pastie.org/private/maxdpn4gmvftzcl2ka0mq

I’m not sure if it will be too complex to the proposal of uniqueness validator but that is a common validation for each system I’ve worked in last years.

WDYT? Something like that should be on the active record or a gem?

Cheers,

Gabriel Sobrinho

gabrielsobrinho.com

Anyone have some opinion?

That is why constraints should be handled by databases in my opinion.

Just create the unique index in the database and write something

like this:

begin

  company.save!

rescue UniqueConstraintException => e # I don't really know the

name for this exception

  render json: {error_message: "Please remove the duplicate

entries."}

rescue e # general rescuer

  log.error "Couldn't create company", e # not sure if that is the

right syntax

  render json: {error_message: "We could't process your request.

Please contact our support team."}

end

Well, at least this has always been my opinion and the reason why I

prefer writing validations in the database using unique constraints or triggers.

Usually the ActiveRecord and Hibernate communities among others

don’t recommend this approach if you intend to support multiple databases, but since I always use PostgreSQL, this is not an issue to me. That is one of the reasons I prefer Sequel over AR. The Sequel community will usually prefer handling validations in the database-level even though you can use validators in Ruby as well to make it easy to handle simple cases but you should always replicate the validations in the database level as well.

Well, that is my opinion since you asked :)

Cheers,

Rodrigo.

Rodrigo,

I’m doing that at database level but I’m not handling the exception because it happens very infrequently.

The point is the validation message that I have to show like any others validations.

Another problem is that I have more than one case like this per form.

I used the partner case but in this same form I have another nested called company activities that can’t repeat the CNAE (an activity means initial date, final date and CNAE).

In this form I have 2 “in-memory” validations but I have others that have more than 5 nested like this.

It will be very painfully if I have to handle each exception to discover what nested failed and add the error by hand.

Like I said before, I see this problem happening over and over again in systems I’ve worked.

But that can be an “enterprise” feature that does not fit to active record proposal.

In that case, I will keep it as a separated gem.

I just want to be sure if I should submit a PR or something like that :slight_smile:

My easiest solution has always been to limit the amount of data in edit mode at the same time. Forms with lots of deeply nested objects are complicated. Using tools like KnockoutJS let me keep the single-page app feel, but help limit the amount of data open in simultaneous edit.

In short, I always push to turn hard problems into easy problems.

In that case I’d handle errors like this in the client-side level besides the database level since I find it simpler, specially if you’re using an OO approach with your models isolated from your views.

In that case, if there is a bug in your client-side code that will

send duplicate entries to the server, in the worst case you’ll get a 500 response if you don’t handle it in the server.

Another approach if you want to handle this in the server-side would

be to do something like this:

begin

  start_database_transaction

  company = Company.create! params[:company] # simplified here, just

create the company with minimal params

  # then add the partners to the company and ask to validate

company. Now the partners should have a valid company_id and the built-in validations should work

  ...

  unless company.valid?

    handle_validation_error company.errors

    raise 'duplicate entries'

  end

  ...

rescue

  rollback_database_transaction

end

Of course the suggestion above would only work with serious

databases (ie, not with the MyISAM engine in MySql, which is the default for older versions of MySql).

In any case I'd suggest to release your gem first and then ask here

if the Rails core would be willing to include it by default in ActiveRecord. In that case they would be able to look at the documentation, source code and tests before making a decision.

But I'd like to tell you why I think lots of people are not using

nested params. If they’re like me the work-flow would look something like this:

The user clicks in a Add Company button and would be asked the

company name. The application creates the company with no partners associated.

Then the interface would add a button "Add partner" asking for the

partner data. In that case your actions would have to deal with a single partner at once, making it easier to work with regular validations and simplifying the logic too.

What I mean is that I find it quite easier to develop applications

using AJAX than trying to create complex forms that do lots of stuff in a single request. Even testing such forms are painful in my opinion. It is much easier to split the application in multiple small parts that are easily testable separately.

Good luck with your decisions,

Rodrigo.

Rodrigo,

This approach seems very interesting but I have one question.

What you do if user create a company and do not fill the partners (i.e.: go to other screen before filling the partners) and use that company on the system?

I also find approaching editing with a view-binding framework like Knockout a bit hard too.

This is an issue I had in the past when I was using KO and I wanted

to be able to handle changes to a template.

The problem is that the changes you make in the text fields will be

immediately reflected in your models before any validations or clicking on a confirm button. What if you click on a cancel button? The model has already changed and the other parts of the application bound to the same model will reflect the changes as you change the inputs too even if you didn’t click in the confirm button.

This is something I think most MVW (model-view-whatever) frameworks

don’t handle well. There should be an easier way to be able to store the changes in a separate model clone until you confirm the changes in which case it should propagate the changes to the actual object.

If you see all examples (specially TODO applications) for KO, Ember,

Angular, Serenade, etc you’ll notice that they won’t provide you “confirm” and “cancel” buttons because that would be much harder to implement properly in all those libraries/frameworks.

How do you handle situations like that with KO?

I can't know the details about your specific applications :wink:

So you're saying that a company is only valid if it has at least one partner, right?

I don't know how "other screens" are reached. Suppose you have a button "Next". It could be disabled until your company has at least one partner.

Other than that you can create an scope "with_partners" or even a default one that will filter companies containing partners only. To get better performance I'd recommend creating the company with an "incomplete" database field set to true. Then, each time you add/remove partners you'd update the field accordingly in an after_save hook for instance.

Would that work for you?

I'm not sure what approach you were talking about, so if you were talking about the server-side handling using a database transaction it would be just a matter of rolling back the transaction if the company doesn't have any valid partner.

Sounds very interesting! :slight_smile:

I can’t change the application right now because we are going to production on next week and I have a bunch of forms like that.

But I’m going to do that on next features and see what happens, thanks! :wink:

About the implementation of validation, we are using that: http://pastie.org/private/osa3pmono5l1ykrd7vipwa

With regards to that, I don't believe it is generic enough for being included in Rails.

It works for your application, but it won't work for nested levels for instance.

Rodrigo,

I don’t get what you mean with nested levels :slight_smile:

A company has many partners. A partner has many credit cards. In both cases you shouldn't have multiple entries (unique restrictions).

This is what nesting is all about.

Rodrigo,

To do that you just need to add the validation to credit card and partner.

Do you mean that? http://pastie.org/private/o5zyoyqsefzhdsmwg3uoq

Yes, I meant that, but I'm not familiar with the way nested params work in Rails. I mean, the only time I really used it was when I wrote an article for Rails Magazine #4, in 2008, so it was a long time ago.

But if you're saying your implementation of NoDuplicationValidator will work for nested cases like that, then I believe on you :slight_smile:

I guess you could create a pull request and it will have a more proper discussion than in this list since people will be evaluating real code.

Now that I'm more aware of what you're accomplishing here, I believe this could be a good addition to AR built-in validators.

It is just that I wasn't much interested at first, not only because I don't use nested params, but also because I'm not using AR either :wink:

Good luck with that :slight_smile: