Help With Sharded Database Setup and Third Party Gem Models

rubyonrails-talk
1

Hello!

I am working on a multi-tenant architecture and would like to take advantage of horizontal scaling with shards.

Approximately 90% of our models should be on per-tenant shards, and the other 10% would be in a ‘global’ context, on the primary shard (things like tenant records, global settings tables, etc.)

So far so good, we can make an abstract_class GlobalRecord that does not specify any connects_to, or specifies connects_to database: { reading: :primary, writing: :primary }. Then, we can define another abstract_class TenantRecord that declares connects_to shards:.

(I can imagine other ways that this would work, e.g. omitting GlobalRecord entirely and just having TenatRecord specify the shard behavior, or having ApplicationRecord specify the shards and GlobalRecord use the connects_to database: configuration).

So far so good.

My big problem is that we have third party gems that define models that inherit directly from ActiveRecord::Base, rather than ApplicationRecord. Generally as far as I am aware it is not possible to alter the ancestry hierarchy of an existing model. Ok, so then we can define connects_to shards: on ActiveRecord::Base so that sharding is the default behavior, and for the 10% case where we do not want sharding I can either fork the library to extend my GlobalRecord class or work around it in some other way.

However, I am finding that ActiveRecord::Base has some special behavior as compared to other abstract classes, in that using connected_to & overwrites the connection stack completely, overwriting GlobalRecord. Thus,

SomeClassThatExtendsTenantRecord.first # All good
SomeClassThatExtendsGlobalRecord.first # All good

ActiveRecord::Base.connected_to(role: :writing, shard: :some_shard) do
  SomeClassThatExtendsTenantRecord.first # All good
  SomeClassThatExtendsGlobalRecord.first # `retrieve_connection': No connection pool for 'GlobalRecord' found for the 'some_shard' shard. (ActiveRecord::ConnectionNotEstablished)
end

The above behavior does not happen if ActiveRecord::Base is swapped out for ApplicationRecord. However, as aforementioned, I am unable to use ApplicationRecord because we have models that need sharding behavior that are from third party gems.

Apologies in advance if I am simply not understanding how to properly use the connected_to api, or if it is assumed that I need to re-establish a connection to GlobalRecord within that block (although I am running into other issues when attempting that, possibly unrelated though. Seems like it should be unnecessary.)

Thanks for any help or suggestions!

John