seed_fu v. populate data from migration

rubyonrails-talk
#1

Why does this migration fail? Using seed_fu was suggested, which I don't
mind, as it looks quite convenient, but I'm scratching my head over the
error:

thufir@ARRAKIS:~/projects/rss2mysql$
thufir@ARRAKIS:~/projects/rss2mysql$ nl db/migrate/*
     1 class CreateUrls < ActiveRecord::Migration
     2 def self.up
     3 create_table :urls do |t|
     4 t.column :url, :string
     5 end
     6 # Url.create(:url => ‘http://www.slashdot.org/index.rss’)
     7 # Url.create(:url => ‘http://groups.google.ca/group/ruby-talk-
google/feed/rss_v2_0_msgs.xml’)
     8 end
       
     9 def self.down
    10 drop_table :urls
    11 end
    12 end
       
    13 class CreateItems < ActiveRecord::Migration
    14 def self.up
    15 create_table :items do |t|
    16 t.column :title, :string
    17 t.column :content, :string
    18 t.column :source, :string
    19 t.column :url, :string
    20 t.column :timestamp, :timestamp
    21 t.column :keyword_id, :integer
    22 t.column :guid, :string
    23 end
    24 end
       
    25 def self.down
    26 drop_table :items
    27 end
    28 end
    29 class CreatePages < ActiveRecord::Migration
    30 def self.up
    31 create_table :pages do |t|
    32 t.references :item
    33 t.column :page, :text
    34 end
    35 end
       
    36 def self.down
    37 drop_table :pages
    38 end
    39 end
thufir@ARRAKIS:~/projects/rss2mysql$
thufir@ARRAKIS:~/projects/rss2mysql$ rake
(in /home/thufir/projects/rss2mysql)
== CreateUrls: migrating

#2

Quoting Thufir <hawat.thufir@gmail.com>:

Why does this migration fail? Using seed_fu was suggested, which I don't
mind, as it looks quite convenient, but I'm scratching my head over the
error:

From http://rails.rubyonrails.org/

reset_column_information()

Resets all the cached information about columns, which will cause them to be reloaded on the next request.

The most common usage pattern for this method is probably in a migration, when just after creating a table you want to populate it with some default values, eg:

class CreateJobLevels < ActiveRecord::Migration
   def self.up
     create_table :job_levels do |t|
       t.integer :id
       t.string :name

       t.timestamps
     end

     JobLevel.reset_column_information
     %w{assistant executive manager director}.each do |type|
       JobLevel.create(:name => type)
     end
   end

   def self.down
     drop_table :job_levels
   end
end

#3

[...]

Which I'll try, but *why* would it fail, given that the table
information has been deleted with the "rake VERSION=0" command, which
dropped the table? There's no table info cached, is there? If so,
where?

-Thufir

#4

[...]

Which I'll try, but *why* would it fail, given that the table
information has been deleted with the "rake VERSION=0" command, which
dropped the table? There's no table info cached, is there? If so,
where?

-Thufir

#5

(in /home/thufir/projects/rss2mysql)
== CreateUrls: migrating

-- create_table(:urls)
-> 0.0417s
rake aborted!
An error has occurred, all later migrations canceled:

uninitialized constant CreateUrls::Url
/home/thufir/projects/rss2mysql/rakefile:9

Do you have a Url model ?

Fred

#6

Quoting Thufir <hawat.thufir@gmail.com>:

> Quoting Thufir <hawat.thu...@gmail.com>:
>
> > Why does this migration fail? Using seed_fu was suggested, which I don't
> > mind, as it looks quite convenient, but I'm scratching my head over the
> > error:
>
> Fromhttp://rails.rubyonrails.org/
>
> reset_column_information()
>
> Resets all the cached information about columns, which will cause them to be reloaded on the next request.
[...]

Which I'll try, but *why* would it fail, given that the table
information has been deleted with the "rake VERSION=0" command, which
dropped the table? There's no table info cached, is there? If so,
where?

The table information has been deleted from the database and the schema
changed, the table classes need to be recreated from the new DB schema. Rails
does not try and modify the model class to to match the new schema, it just
re-creates when you signal that you are done modifying the schema AND that you
want to use the new schema/model.

So the short answer is, contrary to your expectation of how Rails works, the
DB info is effectively cached in the model class's existence.

Jeffrey

#7

Quoting Jeffrey L. Taylor <ror@abluz.dyndns.org>:

Quoting Thufir <hawat.thufir@gmail.com>:
>
>
> > Quoting Thufir <hawat.thu...@gmail.com>:
> >
> > > Why does this migration fail? Using seed_fu was suggested, which I don't
> > > mind, as it looks quite convenient, but I'm scratching my head over the
> > > error:
> >
> > Fromhttp://rails.rubyonrails.org/
> >
> > reset_column_information()
> >
> > Resets all the cached information about columns, which will cause them to be reloaded on the next request.
> [...]
>
> Which I'll try, but *why* would it fail, given that the table
> information has been deleted with the "rake VERSION=0" command, which
> dropped the table? There's no table info cached, is there? If so,
> where?
>

The table information has been deleted from the database and the schema
changed, the table classes need to be recreated from the new DB schema. Rails
does not try and modify the model class to to match the new schema, it just
re-creates when you signal that you are done modifying the schema AND that you
want to use the new schema/model.

So the short answer is, contrary to your expectation of how Rails works, the
DB info is effectively cached in the model class's existence.

If you separated the migration and the population with seed data into two
migrations, the reset_column_info call would not be needed.

Jeffrey

#8

[...]

> uninitialized constant CreateUrls::Url
> /home/thufir/projects/rss2mysql/rakefile:9

Do you have a Url model ?

[...]

Certainly. The rake migration, or migration to version, worked fine
providing no data was written to the db from the migration. In any
event, the models:

thufir@ARRAKIS:~/projects/rss2mysql$ nl url.rb item.rb page.rb
     1 require 'rubygems'
     2 require 'activerecord'

     3 class Url < ActiveRecord::Base
     4 has_many :items
     5 end
     6 require 'rubygems'
     7 require 'activerecord'

     8 class Item < ActiveRecord::Base
     9 has_one :page
    10 belongs_to :feed
    11 validates_presence_of :feed_id
    12 end

    13 require 'rubygems'
    14 require 'activerecord'

    15 class Page < ActiveRecord::Base
    16 belongs_to :item
    17 validates_presence_of :item_id
    18 end
thufir@ARRAKIS:~/projects/rss2mysql$

Although I think I'll change "Url" back to "Feed" and perhaps "Item"
to "Article".

-Thufir

#9

[...]

> uninitialized constant CreateUrls::Url
> /home/thufir/projects/rss2mysql/rakefile:9

Do you have a Url model ?

[...]

Certainly. The rake migration, or migration to version, worked fine
providing no data was written to the db from the migration.

Because when you had those lines commented out, there was no reference
to a Url Ruby class.

In any

event, the models:

thufir@ARRAKIS:~/projects/rss2mysql$ nl url.rb item.rb page.rb
1 require 'rubygems'
2 require 'activerecord'

3  class Url &lt;  ActiveRecord::Base
4   has\_many :items
5  end
6  require &#39;rubygems&#39;
7  require &#39;activerecord&#39;

8  class Item &lt;  ActiveRecord::Base

Well the message is saying that there is no constant named either Url
or CreateUrls::Url defined. This is a standard Ruby message. You
probably know this but, just in case you don't, when you reference a
unprefixed constant name Ruby first looks for it in the root
namespace, and if it doesn't find it it looks in the namespace of the
current Class/Module if there is one.

So I suspect that either:

   1. The automatic loading of ActiveSupport didn't match up Url with
the name of the file containing the class definition for the Url
model, or
   2. The file isn't there at the time you ran the migration because
of a different version control checkout.

I suspect that it's the first assuming that there were no repository
operations in the mean time.

I'm also suspicious of your nl command parameters.

First my version of nl only takes one file name not multiples. Are
there really 3 files being concatenated together before being
numbered? I can't tell. I guess your's does work that way.

Second you seem to be executing this command from the root of the
project directory, and that's not somewhere where activesupport will
find the class files, model class files need to be in under the
apps/models directory structure.

And on my item 2 above, it's generally bad ju-ju to depend on the
external definition of model classes within migrations. Instead you
should create the inside:

  class CreateUrls < ActiveRecord::Migration
        class Url < ActiveRecord::Base
        end

        def self.up
          create_table :urls do |t|
            t.column :url, :string
          end
         Url.create(:url => ‘http://www.slashdot.org/index.rss’)
         Url.create(:url => ‘http://groups.google.ca/group/ruby-talk-
google/feed/rss_v2_0_msgs.xml’)
       end

This isolates the migration from the application code as it changes.
The model will be scoped within the migration class.
The only thing you need to include in the model class for the migration are:
    The fact that it's a subclass of ActiveRecord::Base
     Any association declarations needed by the migration
  and
     any special methods needed by the migration, copied from the real
class (possibly modified) as it needs to be at the state of the
database represented by THIS migration.

But using seeds rather than loading data in a migration is usually
wiser. Data manipulation in a migration should be reserved to do
things like migrating existing data to and from a new schema when the
db can't do it for you automatically.

HTH.

#10

[...]

> Certainly. The rake migration, or migration to version, worked fine
> providing no data was written to the db from the migration.

Because when you had those lines commented out, there was no reference
to a Url Ruby class.

Exactly, just process of elimination that those lines were
problematic.

[...]

Well the message is saying that there is no constant named either Url
or CreateUrls::Url defined. This is a standard Ruby message. You
probably know this but, just in case you don't, when you reference a
unprefixed constant name Ruby first looks for it in the root
namespace, and if it doesn't find it it looks in the namespace of the
current Class/Module if there is one.

So I suspect that either:

1. The automatic loading of ActiveSupport didn't match up Url with
the name of the file containing the class definition for the Url
model, or

See, I was thinking there was a name space problem, but "Url" was
probably a poor choice.

2. The file isn't there at the time you ran the migration because
of a different version control checkout.

I haven't checked this into subversion/etc yet, but I should.

I suspect that it's the first assuming that there were no repository
operations in the mean time.

I'm also suspicious of your nl command parameters.

First my version of nl only takes one file name not multiples. Are
there really 3 files being concatenated together before being
numbered? I can't tell. I guess your's does work that way.

hmm, "nl *" will do the whole directory for me, bog standard ubuntu,
FWIW. Yes, it's a bit vague, so I won't do that in the future.

Second you seem to be executing this command from the root of the
project directory, and that's not somewhere where activesupport will
find the class files, model class files need to be in under the
apps/models directory structure.

Ah, that might very well be the problem -- it's not a rails app per
se, just AR at this stage. I'll probably add rails to it, but may add
a ruby FOX (?) GUI as well or instead.

And on my item 2 above, it's generally bad ju-ju to depend on the
external definition of model classes within migrations. Instead you
should create the inside:

Ok, this is the crux of the issue and the solution for my particular
road block :slight_smile:

class CreateUrls < ActiveRecord::Migration
class Url < ActiveRecord::Base
end

What I did was to add: require 'url'
and that fixed the problem. Although, it's black magic, to me, as to
how ruby found the correct Url class, but "works for now." I don't
like the way the model is defined in multiple places as shown above.
However, please make your case on the bad ju-ju! It's more
conventional to define the model in the migration?

    def self\.up
      create\_table :urls do |t|
        t\.column :url, :string
      end
     Url\.create\(:url =&gt; &#39;http://www.slashdot.org/index.rss')
     Url\.create\(:url =&gt; &#39;http://groups.google.ca/group/ruby-talk-

google/feed/rss_v2_0_msgs.xml')
end

This isolates the migration from the application code as it changes.

But the model is "defined" twice, sorta.

The model will be scoped within the migration class.

yes.

The only thing you need to include in the model class for the migration are:
The fact that it's a subclass of ActiveRecord::Base
Any association declarations needed by the migration

would you expand? I'm not sure what an association declaration is.

and
any special methods needed by the migration, copied from the real
class (possibly modified) as it needs to be at the state of the
database represented by THIS migration.

copied? That sounds like a maintenance nightmare...?

In "real apps" are you really mucking with the db that much with
migrations? It just *sounds* fragile, as described above.

But using seeds rather than loading data in a migration is usually
wiser.

Right, for now I think I'll move on, but will come back to seeds.

Data manipulation in a migration should be reserved to do
things like migrating existing data to and from a new schema when the
db can't do it for you automatically.

HTH.

--
Rick DeNatale

Yes, very helpful. From my perspective, it seems like the black-box
magic which ruby and rails perform is sometimes inconsistent. Why
does *only* that one migration, which populates the db, require an
explicit class definition or import while the other migrations don't?

Actually, it's working now (I think) so I'm just curious on a few
points.

thanks to all,

Thufir

#11

Ah, that might very well be the problem -- it's not a rails app per
se, just AR at this stage. I'll probably add rails to it, but may add
a ruby FOX (?) GUI as well or instead.

Does that mean that ActiveSupport::Dependencies.load_paths is not
configured ? (if you're not loading rails itself and you're not doing
this explicity then the answer is no). What the setting does is tell
ActiveSupport where to look for files when it needs to find a not yet
loaded class.

> And on my item 2 above, it's generally bad ju-ju to depend on the
> external definition of model classes within migrations. Instead you
> should create the inside:

Ok, this is the crux of the issue and the solution for my particular
road block :slight_smile:

> class CreateUrls < ActiveRecord::Migration
> class Url < ActiveRecord::Base
> end

What I did was to add: require 'url'
and that fixed the problem. Although, it's black magic, to me, as to
how ruby found the correct Url class, but "works for now." I don't
like the way the model is defined in multiple places as shown above.
However, please make your case on the bad ju-ju! It's more
conventional to define the model in the migration?

Yes. The problem with using the external definition of the model is
that when you deploy your code, the code all gets deployed in one go,
but your migrations get run one by one. For example, in migration 3
you might decide to add a new column to Url and add a validation on
that column. Now if you deployed your code and tried to run migration
1 it would fail, because even though the database does not yet have
that column Rails would still try to run a validation against it.

Fred