Rails habtm grief am I dumb or rails bug? (desperate ; )

rubyonrails-talk
1

Greetings all,

  I'm having a weird problem with a habtm relationship and honestly I'm beginning to think I may have stumbled upon some weird bug in rails 3. Surely I'm crazy though. I've been beating my head against the wall on this for 3 days, have googled everything under the sun I can think of and still can't come up with an answer.

Ok, the situation:

I'm creating a Rails app to replace both a Java app and a PHP app (java application and php front-end). This is going to be a phased operation with the first phase being the Rails application takes over registration and billing. In order to do this, the Rails application must create data in the databases for the Java and PHP apps. The Rails application itself is using Devise for authentication.

In database.yml I have my standard 3 databases defined and also a connection defined for the Java apps database. Here are pieces of the model definitions for the external object (I'm just creating regular rails models to talk to the external databases):

class Pushbroom::UserAccount < ActiveRecord::Base   require 'digest/md5'   require 'base64'

  establish_connection :pushbroom   set_table_name :user_account   set_primary_key :id

  has_and_belongs_to_many :user_roles, :join_table => 'pb_prod.users_roles', :class_name => 'Pushbroom::UserRole', :foreign_key => 'user_account_id', :association_foreign_key => 'user_role_id'   belongs_to :user, :dependent => :destroy

attr_accessible :user_roles, :admin_notes, :enabled, :username, :password_hash, :prefStore, :accepted_tos, :do_not_contact

end

class Pushbroom::UserRole < ActiveRecord::Base

  establish_connection :pushbroom   set_table_name :user_role   set_primary_key :id

  has_and_belongs_to_many :user_accounts, :join_table => 'pb_prod.users_roles', :class_name => 'Pushbroom::UserAccount', :foreign_key => 'user_role_id', :association_foreign_key => 'user_account_id'

end

And finally my Rails application user object:

class User < ActiveRecord::Base

  before_validation :capture_plaintext_password, :on => :create   before_save :create_pushbroom_user_data   after_save :send_welcome_email

  # Include default devise modules. Others available are:

# :token_authenticatable, :encryptable, :confirmable, :lockable, :timeoutable and :omniauthable   devise :database_authenticatable, :registerable,          :recoverable, :rememberable, :trackable, :validatable

  belongs_to :pb_user_account, :class_name => "Pushbroom::UserAccount", :foreign_key => "pb_user_account_id", :dependent => :destroy, :autosave => true

  # Setup accessible (or protected) attributes for your model

attr_accessible :first_name, :last_name, :username, :dob, :email, :password, :password_confirmation, :remember_me

  validates_presence_of :first_name, :last_name, :username, :dob   validates_date :dob, :on_or_after => lambda { 100.years.ago }, :on_or_after_message => "must be on or after #{100.years.ago.strftime('%m-%d-%Y')}"   validates_date :dob, :on_or_before => lambda { 13.years.ago }, :on_or_before_message => "must be on or before #{13.years.ago.strftime('%m-%d-%Y')}"

  def capture_plaintext_password     @plaintext_password = self.password   end

  def create_pushbroom_user_data     pb_user = create_pushbroom_user     add_trial_subscription_to_pb_user(pb_user)     pb_user_account = create_pushbroom_user_account(pb_user)     add_subscription_plan_roles_to_pb_user_account(pb_user_account)     self.pb_user_account = pb_user_account   end

  def create_pushbroom_user     pb_user = Pushbroom::User.new     pb_user.attributes = self.attributes.slice(       "email",       "first_name",       "last_name",       "dob")

    pb_user   end

  def add_trial_subscription_to_pb_user(pb_user)     subscription = Pushbroom::Subscription.new

subscription.populate_from_subscription_plan(Pushbroom::SubscriptionPlan.find_by_name("TRIAL"))     pb_user.subscriptions << subscription   end

  def add_subscription_plan_roles_to_pb_user_account(pb_user_account)     roles_granted = pb_user_account.user.subscriptions.first.subscription_plan.roles_granted     pb_user_account.user_roles = roles_granted   end

  def create_pushbroom_user_account(pb_user)     pb_user_account = Pushbroom::UserAccount.new     pb_user_account.enabled = true     pb_user_account.password_hash = Pushbroom::UserAccount.create_password_digest(@plaintext_password, self.username)     pb_user_account.username = self.username     pb_user_account.user = pb_user

    pb_user_account   end

  def send_welcome_email     AccountMailer.welcome(self).deliver   end end

Seems like it should be pretty vanilla. The ONLY weirdness here is that they aren't in the native rails database and one of the fields is named funny in the relations table.

So here's a rails console session where I create a rails user, call the method to create the external objects, then try to save:

ruby-1.9.2-p180 :001 > def user_fred ruby-1.9.2-p180 :002?> { ruby-1.9.2-p180 :003 > :first_name => "Fred", ruby-1.9.2-p180 :004 > :last_name => "Flinstone", ruby-1.9.2-p180 :005 > :username => "fflint", ruby-1.9.2-p180 :006 > :dob => "1986-06-01", ruby-1.9.2-p180 :007 > :email => "fred@mydomain.org", ruby-1.9.2-p180 :008 > :password => "badpass" ruby-1.9.2-p180 :009?> } ruby-1.9.2-p180 :010?> end => nil ruby-1.9.2-p180 :011 > user = User.new(user_fred) => #<User id: nil, email: "fred@mydomain.org", encrypted_password: "$2a$10$IiEOEoSnXIrP7VJAQYckfOVXuzm7Y5ZGo20ayLpSkHhz...", reset_password_token: nil, remember_created_at: nil, sign_in_count: 0, current_sign_in_at: nil, last_sign_in_at: nil, current_sign_in_ip: nil, last_sign_in_ip: nil, created_at: nil, updated_at: nil, first_name: "Fred", last_name: "Flinstone", username: "fflint", dob: "1986-06-01", pb_user_account_id: nil> ruby-1.9.2-p180 :012 > user.create_pushbroom_user_data => #<Pushbroom::UserAccount id: nil, created_by: nil, created_at: nil, updated_by: nil, updated_at: nil, admin_notes: nil, enabled: true, username: "fflint", password_hash: "blah blah", user_id: nil, prefStore: nil, accepted_tos: nil, do_not_contact: nil> ruby-1.9.2-p180 :013 > user.pb_user_account.user_roles => [#<Pushbroom::UserRole id: 1, created_by: "script", created_at: "2008-11-10 12:10:44", updated_by: "script", updated_at: "2008-11-10 12:10:44", admin_notes: "", name: "user", description: "Generic User Role", conditional: false>] ruby-1.9.2-p180 :014 > user.save! NoMethodError: undefined method `relation' for nil:NilClass   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activesupport-3.0.5/lib/active_support/whiny_nil.rb:48:in `method_missing'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/arel-2.0.9/ lib/arel/insert_manager.rb:22:in `insert'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/arel-2.0.9/ lib/arel/crud.rb:26:in `insert'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/associations/ has_and_belongs_to_many_association.rb:76:in `insert_record'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/associations/association_proxy.rb: 151:in `send'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/autosave_association.rb:306:in `block in save_collection_association'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/associations/ association_collection.rb:431:in `block in method_missing'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/associations/association_proxy.rb: 216:in `block in method_missing'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/associations/association_proxy.rb: 216:in `each'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/associations/association_proxy.rb: 216:in `method_missing'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/associations/ association_collection.rb:431:in `method_missing'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/autosave_association.rb:297:in `save_collection_association'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/autosave_association.rb:163:in `block in add_autosave_association_callbacks'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activesupport-3.0.5/lib/active_support/callbacks.rb:415:in `_run_create_callbacks'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/callbacks.rb:281:in `create'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/persistence.rb:246:in `create_or_update' ... 18 levels...   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/callbacks.rb:277:in `create_or_update'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/persistence.rb:56:in `save!'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/validations.rb:49:in `save!'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/attribute_methods/dirty.rb:30:in `save!'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/transactions.rb:245:in `block in save!'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/transactions.rb:292:in `block in with_transaction_returning_status'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/connection_adapters/abstract/ database_statements.rb:139:in `transaction'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/transactions.rb:207:in `transaction'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/transactions.rb:290:in `with_transaction_returning_status'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ activerecord-3.0.5/lib/active_record/transactions.rb:245:in `save!'   from (irb):14   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ railties-3.0.5/lib/rails/commands/console.rb:44:in `start'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ railties-3.0.5/lib/rails/commands/console.rb:8:in `start'   from /Users/gander/.rvm/gems/ruby-1.9.2-p180@sms2/gems/ railties-3.0.5/lib/rails/commands.rb:23:in `<top (required)>'   from script/rails:6:in `require'   from script/rails:6:in `<main>'ruby-1.9.2-p180 :015 >

If I remove the role assignment, everything is just peachy (finds, saves, destroys, etc), but the second I try to save roles everything blows sky-high with this message that, frankly, I don't get. It knows its got the roles, there is no nil object that I can tell. . .and basically if I wasn't already bald I'd be pulling my hair out ; )

Any insight into this is EXTREMELY appreciated!

Gerald

2

If I remove the role assignment, everything is just peachy (finds, saves, destroys, etc), but the second I try to save roles everything blows sky-high with this message that, frankly, I don't get. It knows its got the roles, there is no nil object that I can tell. . .and basically if I wasn't already bald I'd be pulling my hair out ; )

A cursory examination of the arel code suggests this might happen if one of the columns you name doesn't in fact exist

Fred

3

Fred, thanks for the reply. That would seem to be logical and one of the first things I checked. The table definition for the relationship table is (pulled straight from the production db):

CREATE TABLE IF NOT EXISTS `users_roles` (   `user_account_id` bigint(20) NOT NULL,   `user_role_id` bigint(20) NOT NULL,   KEY `FKF6CCD9C617041664` (`user_role_id`),   KEY `FKF6CCD9C63044D5F0` (`user_account_id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Any other ideas (or do you see something I don't)? This one is driving me nuts and has me totally shut down : /

G

4

Another note along these lines. . . if I load an existing user with existing roles, they load fine. It just won't let me add new ones.

G

5

Fred, thanks for the reply. That would seem to be logical and one of the first things I checked. The table definition for the relationship table is (pulled straight from the production db):

CREATE TABLE IF NOT EXISTS `users_roles` ( `user_account_id` bigint(20) NOT NULL, `user_role_id` bigint(20) NOT NULL, KEY `FKF6CCD9C617041664` (`user_role_id`), KEY `FKF6CCD9C63044D5F0` (`user_account_id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Any other ideas (or do you see something I don't)? This one is driving me nuts and has me totally shut down : /

I'd double check the sql statemnts rails generates when it is listing columns to see if they're going to the right database. Have you tried stepping though the rails insert process in the debugger to see where it dies?

Fred

6

Ok, so I've never put a debugger on a rails app before, wasn't real excited to do so ; )

But. . I did. I'm still poking around, BUT it LOOKS as though it MAY (like all the emphasis?) not know that the users_role table is in pb_prod (external db) even though it DOES know that the two ActiveRecord objects are and there's explicit declaration in my habtm key definitions. Details:

in arel-2.0.9/lib/arel/table.rb:101   It's returning false for table_exists? (well, returns nil because of that) That's calling table.rb:121 which says:   @table_exists? ||= tables.key?("pb_prod.users_roles") || engine.connection.table_exists?("user_account_id")

Now, to me it looks like the values for those two or backwards (key implies key, not table name, and reverse for table_exists?) but. . key? returns false, and table_exists? sends me to:

activerecord-3.0.5/lib/active_record/connection_adapters/abstract/ schema_statements.rb:20

      def table_exists?(table_name)         tables.include?(table_name.to_s)       end

And, don't you know it, tables contains the rails database tables not the external application's tables.

So. . it IS looking at the wrong database.

Ok, now what? I've tried explicitly stating in the key definitions for the habtm what schema to look at. Any suggestions on what to look at from here?

Thanks again!

G

7

Grr, have had two meetings in the middle of all this, forgive my lack of coherence. Bottom line is it looks like it instantiates the engine as the rails database and with no way (that I know of) of specifying which db to look at for the relation table I'm kind of screwed. It seems to ignore the fact that I'm using :join_table => 'pb_prod.users_roles" for the relationship definition (pb_prod is the external database) and isn't picking up which database to use from either object (connection :pushbroom).

So it looks to me, at the moment, as though I'm stuck. It does bother me though that it LOADS the relationships fine. Just seems to be a problem on inserts.

Again, just spouting what I've found and desperately looking for a solution. Would prefer to find out I'm just being an idiot instead of having to implement some solution with seriously high code smell.

Thanks! G

8

Short of finding and squashing the bug, one not-too-smelly alternative might be to make a real model for users_roles, like this:

class UsersRoles < ActiveRecord::Base   establish_connection :pushbroom   set_primary_key nil

  belongs_to :user_account, :class_name => 'PushBroom::UserAccount'   belongs_to :user_role, :class_name => 'PushBroom::UserRole' end

then you could replace the habtm call with a set of has_many :throughs, which shouldn't have the same problem (since you've explicitly told Rails the join table is on the :pushbroom connection).

--Matt Jones

9

Good timing, just got it fixed (non-smelly)and you're pretty close to the mark.

The answer? has_many :through - something I'd never taking much of a look at, but it is actually a pretty nice feature (even in other circumstances).

Basically this just allows me to create a model class which represents the relationship. And since I have a model class for it I can explicitly specify the database to connect to.

For posterity sake, here's the code:

class Pushbroom::UsersRolesRelationship < ActiveRecord::Base

  establish_connection :pushbroom   set_table_name :users_roles

  belongs_to :user_account   belongs_to :user_role end

class Pushbroom::UserAccount < ActiveRecord::Base

  establish_connection :pushbroom   set_table_name :user_account   set_primary_key :id

  has_many :users_roles_relationships   has_many :user_roles, :through => :users_roles_relationships, :source => :user_role end

class Pushbroom::UserRole < ActiveRecord::Base

  establish_connection :pushbroom   set_table_name :user_role   set_primary_key :id

  has_many :users_roles_relationships   has_many :user_accounts, :through => :users_roles_relationships, :source => :user_account end

And is used thusly:

def add_subscription_plan_roles_to_pb_user_account(pb_user_account)     roles_granted = pb_user_account.user.subscriptions.first.subscription_plan.roles_granted     pb_user_account.user_roles = roles_granted end

Thanks a ton folks for helping me get this train moving again! All my tests are passing and it seems to be working, but if you see something wrong, please do still let me know.

Thanks! Gerald

10

Oh, also meant to say I don't know that the habtm thing is a bug (not a feature either : b). It's not the way I would like to see it work but the rails code that I looked at just flat doesn't make allowances for m2m relationship tables outside of the rails db that I could tell.

Gerald