Support the ability to write properties within models

rubyonrails-core
1

Why not support the ability to write properties within models?

After two years of trying I left mongoid to return to activerecord. The reason why I came back to activerecord is that I realized I had chosen MongoId just for the chance to express attributes in the model itself and not for the advantages offered by mongodb.

I suppose this is a fairly common scenario, as well as the reason for the adoption of other orms as DataMapper: I think that many of those who choose these ORMs, often do so because they are attracted by the possibility of writing the model classes in the classical way and not because they consider truly worst ActiveRecord pattern.

It is not just a matter of style, what you can do with the approach used by DataMapper or MongoId is something you can just not express with ActiveRecord, an example over all, how could you do that in ActiveRecord?

module Profile   extend ActiveSupport::Concern   included do     field :name     field :age, :as => :integer

  end end

class User < ActiveRecord::Base   include Profile

end

class Player < ActiveRecord::Base   include Profile

end I wondered how we could achieve the same behavior with ActiveRecord without altering its nature and in response I created this gem ActiveRecordSchema (https://github.com/mcasimir/active_record_schema).

I have no interest in promoting my work I just want to propose that a similar procedure can be adopted directly in Ruby on Rails

I do not know if a similar feature has already been taken into account, in which case I'm sure you had good reasons for not adopting it

2

Seems like unnecessary duplication if you have to specify the field in a migration as well as in the model itself.

What’s the point of having to specify the field on the model if it’s already in the table? Unless there is some other benefit that I am not seeing right away.

3

Clearly this isn’t the ActiveRecord approach, where what is in the table is the authoritative view. If you want to use ActiveRecord in a style where the models contain the authoritative view, check out the third-party library HoboFields:

http://cookbook.hobocentral.net/manual/hobo_fields

It gives you a DSL to specify the attributes on a model, and will automatically generate database migrations to make your database reflect this state.

-Andrew.

4

There are incredible benefits to separate migrations from the model. Also note that it is not so obvious that the procedural code to write and track changes to the database code is 'duplicate' of what defines the abstract structure of application domain.

Advantages, some philosophical, some more practical:

1) Abstraction     Think in terms of classes, modules, attributes and relationships when designing the model, think in term of columns and tables when migrating database

2) Switch from/to different ORMs (direct consequence of point 1)

3) Faster development (write only 'has_and_belongs_to_many' instead 'create_table ...' 4) Probably the most significant: reuse of code.

about the reuse of code I just can not understand how this failure has not been felt in all these years, and this is the only reason that makes me doubt that there are deep reasons for not adopting such a solution.

A use case: Imagine you write an application with a user model. In time, our user model is refined until it becomes almost perfect so is it right that we would like to reuse the same scheme for other applications because it is so good. Suppose you have to do this for a class Player.

It would be so uncomfortable that we resort to definitely do a copy-paste from schema.rb in a brand new migration, changing 'user' with 'player'

And if we would distribute our schema? We need to create a generator for that. Now check the difference:

module Profile    extends ActiveSupport::Concern

   included do      field :name      field :age, :integer      # ..and so on    end end

class Player < ActiveRecord::Base      include Profile end

rails g migration create_players --from Player

done!

There are thousand of advantages on doing so, but I do not wanna waste your time anymore ;).

But please check my implementation here: https://github.com/mcasimir/active_record_schema and think how useful it can be if gems such as paperclip or friendly_id were be supported by this behaviour here:

gem 'friendly_id_schema', :git => 'git://gist.github.com/2629071.git' gem 'paperclip_schema', :git => 'git://gist.github.com/2629431.git'

Plus note that you can distribute schemas in a while using microgems.

5

I've checked that gem before to implement my third-party solution.. https://github.com/mcasimir/active_record_schema, it's very similar to hobo_fields.

I'm not intrested in promote it but to encourage you to at least be doubtful that such a solution is to consider more calmly.

Think of the benefits: easier to create reusable code = more gems, easier to switch from other frameworks or ORMS to activerecord, faster and less error-prone development (you can focus on the business first and no the dbms later)

and although this will probably not affect you: after spending 2 years monkey-patching mongoid rather i will not spend the next 2 to patch activerecord :wink:

One last word.. i know this is a little different from what strictly is ActiveRecord approach but it is still ActiveRecord! I always repeat myself to focus on the principles not on rules.

With a few line of codes in active_record/migration generator you can achieve a lot of benefits.. would you really throw them away to follow a rule?

Thank you very much, i really appreciate your work and your understanding Best regards

Maurizio

6

No other answer? I do not have you convinced yet then … I hope that does not bother you if I try again. About ‘this is not active record’: with ruby ActiveRecord you have improved the pattern yet mixing it with STI and supporting associations with macros. About ‘this is duplicated code’: what about STI? where in the migrations code or in schema.rb or even in the database itself persist the information that an attribute belongs to a children class and not on the whole hierarchy classes? And where in the migration code is eg class Content < AR::Base class Article < Content class Video < Content create_table content add_column :title … add_column :url … add_column :text …
Here Video instances will respond to :text, and Article will respond_to :url without batting an eye. moreover even in the schema.rb the ‘contents’ table schema is all you can find… perhaps this is a consequence of ActiveRecord not yet supporting STI very well. This can’t be the same of: class Content < AR::Base field :title class Article < Content field :text class Video < Content field :url there is a very important information here: text is something that belongs to Article, and url is something that belongs to Video. This way you could also enforce that Video should not respond to ‘text’ and Article should not respond to ‘url’. This is nothing you can express with migrations! Migrations are only a history to keep track of changes you made to database. They are not good to express the logical structure of the application domain.

All that i ask to you is what you think about that? But answer without focusing on my purposed solution, but in a more abstract way. I think the topic of discussion here is not whether to make a change ​​or not to ActiveRecord … I probably said bad in this regard… but I’d like to know what you think of this lack… first of all… do you think this is so? have you ever thought of it?

For me this was the first thing I noticed when i approached ruby on rails from java world. This leak significantly restricts the ability to refactor, mantain, document and reuse code, in other word a weakness in dealing with changes over time. Why do not just try to make things better? You have just done it in past, em i wrong?

Maurizio

7

Shouldn't you be using a polymorphic association for those type-specific fields then, rather than STI?

8

Between Content and Article and Content and Video? Something like Content belongs_to content_owner? Probably this is also a solution, anyway this is not the point.

Now, i agree this example is really poor, try to figure out a situation were STI fits better if you like but, rather to debate if STI is a good thing or not.. focus instead on the fact that there are things we can't just say with plain migrations alone.

Another idea you've just inspired: what about this?

class Content    inheritable :strategy => :mti    field :title

class Article    field :text

now running rails g migration create_articles --from Article

will generate the necessary migration to do what you just said but without distorting the *logical* structure: an Article is a Content, and a Video is a Content too. What you would do with association is to say 'Article' has fields of a contents and Video has fields of a content as well. But this is database centric and this is just the kind of coupling that causes pain in the long run.

This is another clue that we need a level of abstraction here.

9

I wouldn’t go for this route when trying to convince one.

IMO database migration and the ORM are different concerns and we

should actually use the single responsibility principle with regards to them.

For example, I prefer to maintain my database migrations in a

different project using standalone-migrations:

https://github.com/thuss/standalone-migrations

There are advantages and disadvantages using this approach, but I

still think it is a better option than integrating the migrations directly in the application.

Disadvantages:

1- rails g model user email password_hash role:references

This wouldn't be much useful if migrations are not integrated. This

could be worked around in the standalone-migrations if they included some generators to easy migrations generation.

The generated model class itself is so short that it is pretty fast

to create it by hand. With ActiveRecord it is “class MyModel < ActiveRecord::Base; end”. Just replace with Sequel::Model for Sequel.

2 - application-specific classes can't be used inside migrations (is

this really a disadvantage?)

Some people like the idea of updating some data in the database

during a migration by using the ORM itself inside the migrations.

While this can be handy sometimes, it can create some issues too.

For instance, your migration is now relying on your application code

by the time you created the migration.

After a while that code may no longer exist or do a different thing,

so this is pretty dangerous.

I've already experienced some issues in open-source projects where I

couldn’t run a migration because the code inside earlier migrations would no longer run.

The solution provided by Rails is to use "rake db:setup" for fresh

databases, but what if you need to run several migrations because you already have a database, but the code has changed in such a way that some of the earlier migration code won’t run anymore?

That is one of two of the strongest reason I want to keep my

migrations in a separate project. Doing that you’ll be assuring that no application-specific domain code will be part of the migrations.

3 - You can't know what was the specific database version in use

within some commit.

Working around this limitation is pretty simple if you're using Git.

Just use your migrations project as a submodule and every time your classes rely on a different database state, update the commit the submodule points to.

Advantages:

I've talked already about most of them. Here are some more:

1 - You don't risk being unable to migrate or rollback your database

due to some issue in your code

I want to be sure that I'll be able to quickly run my migrations or

rollback the database even if my application won’t boot for some reason, like a bad initializer that relies on some network connection that is not available or anything else. I want to just keep it simple.

2 - You can have different applications in possibly different

languages interacting with your database while having a single separate place to store the migrations.

Back to the subject:

I've tried to convince you not to argue about defining the

properties inside the models to avoid duplication of code in migrations because they should be really isolated from each other

Now, I'd like to share that I like the ability to quickly see what

are the column names for some domain by looking at its source-code. Your solution provides that, but it is not the only way to achieve this goal.

Some people might want to document such columns by using the

attr_protected and attr_accessible methods. I don’t agree that protection against mass-assignment is a responsibility of the models, hence I don’t agree with the default

config.active_record.whitelist_attributes = true in application.rb.

They should be handled by the controller in a MVC or Model2 architecture.

But there is a simple solution that solves this issue for me (this

example assumes Sequel, since that is my ORM of choice):

rails r "puts User.columns.join ', '"

Then it is just a matter of replacing the comment in my classes when

the table changes (I always omit ‘id’):

class User < Sequel::Model

    # columns: email, password_hash, role_id

    ...

end

So, I'd like to ask you how do you think your proposal could help me

if I were using ActiveRecord instead of Sequel, since I’d use the same ideas with AR as well.

Cheers,

Rodrigo.
10

The solution provided by Rails is to use "rake db:setup" for fresh databases, but what if you need to run several migrations because you already have a database, but the code has changed in such a way that some of the earlier migration code won't run anymore?

This is off-topic. You need to define the code you relay on in the migration file

class PostsToDasherize   table_name :posts   def dasherize!     title.dasherize!     title_will_change!     save!   end end

class DasherizePosts < ActiveRecord::Migration   def self.up     ActiveRecord::Base.transaction do       PostsToDasherize.all.each{|p| p.dasherize! } # Use batches instead     end   end

  def self.down     # ...   end end

Robert Pankowecki

11

2 - application-specific classes can’t be used inside migrations (is this really a disadvantage?)

agree.

Some people like the idea of updating some data in the database during a migration by using the ORM itself inside the migrations.

While this can be handy sometimes, it can create some issues too.

For instance, your migration is now relying on your application code by the time you created the migration.

You’re right, i did not put enough attention, this other approach would solve the problem:

class Content < AR::Base

inheritable

…

class Article < Content

rails g migration create_articles --from Article --strategy mti

Back to the subject:

I've tried to convince you not to argue about defining the

properties inside the models to avoid duplication of code in migrations because they should be really isolated from each other from my point of view

I totally agree i don’t was arguing to define properties inside the models to avoid duplication of code… i was just answering to a previous response ;).

What you say is just the same that i say: migrations need to be more isolated from the rest. From your response i guess you have missed the conversation from the begining am i right? My concern was that implementing properties in models ActiveRecord could have the same benefits that some other ORMs are taking advantage yet.

Now, I'd like to share that I like the ability to quickly see what

are the column names for some domain by looking at its source-code. Your solution provides that, but it is not the only way to achieve this goal.

Really… this is not enough… what i want is to have them defined in models… there are many solutions to document schema… not the main point. Availability of attributes in mixins, properly use of inheritance, more indirection… these are benefits we can’t miss. The fact that models are self-documented would be a nice side effect achieved for free.

To see what benefits i’m talking about refer to https://github.com/mcasimir/active_record_schema (yes i’know… ugly name) or to previous messages.

Thanks

Maurizio

12

Back to the subject:

I've tried to convince you not to argue about defining the properties inside the models to avoid duplication of code in migrations because they should be really isolated from each other from my point of view

... From your response I guess you have missed the conversation from the begining am i right?

I've just re-read all conversation to make sure, but I couldn't find out why you guessed so.

My concern was that implementing properties in models ActiveRecord could have the same benefits that some other ORMs are taking advantage yet.

I don't see any issues with the way AR Migrations were designed when used separately from the application itself.

I really don't like the idea of automatically generating migrations from information extracted from the model class. Grails always supported that kind of database evolution, inherited from Hibernate and I always found it a bad approach.

For example, what if you remove an attribute declaration from your model? Maybe you're not using it anymore in your specific application, but you don't want it to generate a migration that will remove that field from the table.

Also, how can you detect field renaming, for example? This is too much to automate in my opinion. I prefer to be explicit in generating my migrations than to try to automatically generate one based on current database schema and annotated model class.

When you have multiple applications using the same database it is likely that not all applications will need all fields from the database, so it makes sense to document just those who are relevant to your specific application.

Also, when you want to create a program to interact with a legacy database, it is great that you're able to use it without having to declare all fields in the model and just rely on database introspection made by ActiveRecord or Sequel on class generation.

So, for my use case I can't really see any advantages in enforcing annotations in ORM models. And to be honest, I have a hard time trying to figure out in what use case such annotated classes could be useful...

You can already use database introspection within mixins as well as setting up inheritance. Maybe it would help if you could provide some concrete examples on what you think it is currently impossible to do with ActiveRecord and how that would become possible with your approach.

Best, Rodrigo.

13

Yes, i'm sorry .. examples.. anyway: declaring fields in models should not be mandatory! To keep track of renaming, changing and deleting of columns is already up to the migration system. We could think to a rake task to check if everything is up-to-date.. (few line of code) but i don't care too much of it because i think such kind of changes are something that is better to do by hand.

The generative approach is something that is needed only for the non-destructive changes (adding column, tables and indexes) IMO. Also you should see the generator for what it is: just something that can write for you something that you should write by yourself otherwise.

But here is an example of what misses the migration-alone approach:

# paperclip_schema gem:

module PaperclipSchema   extend ActiveSupport::Concern

  module ClassMethods

    def has_attached(name, options = {})

      has_attached_file(name, options)              field :"#{name}_file_name"       field :"#{name}_content_type"       field :"#{name}_file_size", :as => :integer       field :"#{name}_updated_at", :as => :datetime

   end

  end end

ActiveRecord::Base.send :include, PaperclipSchema # ~

Application models:

class Picture    has_attached("image") end

class User   has_attached("avatar") end

now generate migrations, and enjoy using Paperclip (again this is just an very simple example)..

Application Logic is not aware of what happens below and it seems correct to me. How can't you see this is a level of abstraction we don't have now?

As a counterevidence: how much time will you need to port your code to another ORM.. such as DataMapper.. or switch from DM to ActiveRecord, or even MongoId (supposing you wish also to change database)? This way they would be almost interchangeable.

Rails is trying to move to an ORM agnostic model or em i missing something? Well this is a way to make ActiveRecord play nice in the ORM agnostic world.

14

Yes, i'm sorry .. examples.. anyway: declaring fields in models should not be mandatory! To keep track of renaming, changing and deleting of columns is already up to the migration system. We could think to a rake task to check if everything is up-to-date.. (few line of code) but i don't care too much of it because i think such kind of changes are something that is better to do by hand.

The generative approach is something that is needed only for the non-destructive changes (adding column, tables and indexes) IMO. Also you should see the generator for what it is: just something that can write for you something that you should write by yourself otherwise.

But here is an example of what misses the migration-alone approach:

# paperclip_schema gem:

module PaperclipSchema    extend ActiveSupport::Concern

   module ClassMethods

     def has_attached(name, options = {})

       has_attached_file(name, options)

       field :"#{name}_file_name"        field :"#{name}_content_type"        field :"#{name}_file_size", :as => :integer        field :"#{name}_updated_at", :as => :datetime

    end

   end end

ActiveRecord::Base.send :include, PaperclipSchema # ~

Application models:

class Picture     has_attached("image") end

class User    has_attached("avatar") end

now generate migrations, and enjoy using Paperclip (again this is just an very simple example)..

I really don't like this approach. As I've said before, I don't like the idea of mixing database evolution code with the application code itself. And I don't think there is a need for that.

Traditionally, gems like Devise and similar ones will add generators to your application to create the missing columns in the affected models so that you have the chance to review and understand what columns are expected to exist in each model table.

And I don't see any problem with that approach. I don't really think we should try to make migrations a bit easier by providing some annotations to the model classes.

Application Logic is not aware of what happens below and it seems correct to me. How can't you see this is a level of abstraction we don't have now?

I can see it as a level abstraction we don't currently have. I just don't find it useful enough to worth it.

As a counterevidence: how much time will you need to port your code to another ORM.. such as DataMapper.. or switch from DM to ActiveRecord, or even MongoId (supposing you wish also to change database)? This way they would be almost interchangeable.

I've replaced recently my models from ActiveRecord to Sequel and didn't experience any issues during this transition. And my migrations weren't affected at all.

Rails is trying to move to an ORM agnostic model or em i missing something? Well this is a way to make ActiveRecord play nice in the ORM agnostic world.

Why do you think Rails would be more ORM agnostic than it already is just because you're introducing annotations to model classes?

I am already able to use Sequel with Rails and I'm not even using some sequel-rails gem for doing that. So I guess Rails is already ORM agnostic enough in my opinion.

Best, Rodrigo.

15

I understand your opinion but i'm not introducing, and not inventing nothing new. Plus Sequel is active record (the pattern not the library).. just try with mongoid or mongomapper..

Rails will get more orm agnostic because you will have the ability to switch between orms that already support that interface..

Ruby on Rails will never be really orm agnostic untill we need to write 2 separate gem, one for active_record (with gererators) and one for mongoid (with fields expressed in models), for any model related feauture. Can you agree now?

Migrations are meaningless outside active record (again the pattern not the gem).. we can't enforce a migration centered approach in an ORM agnostic environment.. perhaps to me it seems that is more comfortable to make ActiveRecord to support properties inside models that to ask Mongoid and other 500 ORMs around to dump fields and support migrations.

To be even more clear on what ORM-agnostic means: try to ship a Gem for both Mongoid and ActiveRecord without have somewhere in your code an 'if is Activerecord do ... else if do ...'

Now if you don't like this approach.. again.. there is no need for this to be mandatory.. an ORM agnostic model implementation needs only to support it, just in case a Gem would require.

Considering that ActiveRecord is the most supported ORM by community, without this we never.. and i repeat *never* will really see the benefit of ORM-agnostic models. Gem developers will have to write the same code twice forever or ship it with proprietary adapters in place of the level of abstraction that is missing here. See Devise, as an example, that has dropped schema generation to non migration based ORMs, or Carrierwave that has adapters as external gems.

16

This statement is not factual. Migrations, in the sense of being any systematic method of aggregating changes to a database schema over time and generating the necessary DDL, certainly can possess meaning outside of whatever method of data representation (AR, DM, IM, QO, ad nauseam) is chosen by an OO system designer.

It is entirely conceivable that two or more separate projects employing AR (the pattern) could reference the same database and table and each consider only the subset of the schema necessary for their specific function. In such a case, under your proposal, which implementation should provide the migration?

17

I would love to see the properties available on that specific Ruby model defined by annotations–much like relationship methods define what relationships are available on any given model. But I do not believe that this should flow backward into generation migrations.

If the model and the migration/database state can clearly had different meaning/values (and it’s incredibly obvious that they can especially in the case of more than one app accessing the same database) it makes sense to separate the idea of what attributes are available on the model from what attributes are in the database.

It has the added benefit of making it easy to determine the properties to which you have access while working on a model without needing to reference the schema.

James Coleman

18

So, if I understand correctly, the proposal would be to just add some annotation method to make it explicit which columns our model cares about using a common API so that other gem authors could use that information for doing something meaningful.

If that is the proposal, I'd vote for it. The annotations would only store the relevant column names so that they can be queried by third-party gems and would bring a documentation of available/relevant fields of the model as a side effect.

Nice :slight_smile:

Good luck on trying to convince the core members about this idea :slight_smile:

Cheers, Rodrigo.

19

Migrations are meaningless outside active record (again the pattern not the gem).. we can't enforce a migration centered approach in an ORM agnostic environment.. perhaps to me it seems that is more comfortable to make ActiveRecord to support properties inside models that to ask Mongoid and other 500 ORMs around to dump fields and support migrations.

This statement is not factual. Migrations, in the sense of being any systematic method of aggregating changes to a database schema over time and generating the necessary DDL, certainly can possess meaning outside of whatever method of data representation (AR, DM, IM, QO, ad nauseam) is chosen by an OO system designer.

Yes, this is true, i was trying to say that rely on migrations to achieve ORM-agnostic model needs to rely on the fact that all the ORMs around should use them, or em I wrong?

It is entirely conceivable that two or more separate projects employing AR (the pattern) could reference the same database and table and each consider only the subset of the schema necessary for their specific function. In such a case, under your proposal, which implementation should provide the migration?

The same way is provided now. My current implementation is conservative over the AR migration system, so no problem here. I don't want drop migrations for a better system.

But I understand probably here can be a tricky part:

User (App a)    name

User (App b)    name

Now say that we want to add "age" to User for the app B. Here we could add the property on the model and safely run the migration generator. The migration generator will take into account the current state of the database, so no problem here until you don't want it to support also irreversible changes, a bad idea for me.

The only case in which the generator fails to generate is when the state of the database is not updated respect to migrations, but no problem arises if we use two different models refering to same table, only the differences will be taken into account. Was this your question?

Regards Maurizio

20

This is ok also to me, in fact the generator is not necessary, the big goal to achieve is to have the api to annotate models.. after all the subject of discussion is 'Support the ability to write properties within models'