Switch from Makara to Rails Multiple Databases

rubyonrails-talk
1

Hey,

I’m currently working on a mid-size app where we have a primary DB with two replicas. They’re on AWS RDS and we currently use the https://github.com/instacart/makara gem to do the heavy lifting for multi-db support. We would like to switch to the inbuilt Rails Multiple DB support if it’s easily configurable, but I’m having a hard time figuring out the proper configuration.

This is the current config/database.yml for production:

production:
  <<: *default

  makara:
    id: mysql
    connections:
      - role: master
        host: <%= ENV.fetch("DATABASE_HOST") %>
        name: Master
      - role: slave
        database: <%= ENV["DATABASE_REPLICA1_NAME"] %>
        username: <%= ENV["DATABASE_REPLICA1_USERNAME"] %>
        password: <%= ENV["DATABASE_REPLICA1_PASSWORD"] %>
        host: <%= ENV["DATABASE_REPLICA1_HOST"] %>
        name: Replica 1
      - role: slave
        database: <%= ENV["DATABASE_REPLICA2_NAME"] %>
        username: <%= ENV["DATABASE_REPLICA2_USERNAME"] %>
        password: <%= ENV["DATABASE_REPLICA2_PASSWORD"] %>
        host: <%= ENV["DATABASE_REPLICA2_HOST"] %>
        name: Replica 2

And it just works.

So far, all the examples and the Rails guides work with one replica.

My attempt for the migration to the inbuilt Rails multi db support would look something like this in the config/database.yml:

production:
  <<: *default
  primary:
    database: <%= ENV.fetch("DATABASE_NAME") %>
  primary_replica_one:
    database: <%= ENV["DATABASE_REPLICA1_NAME"] %>
    username: <%= ENV["DATABASE_REPLICA1_USERNAME"] %>
    password: <%= ENV["DATABASE_REPLICA1_PASSWORD"] %>
    host: <%= ENV["DATABASE_REPLICA1_HOST"] %>
  primary_replica_two:
    database: <%= ENV["DATABASE_REPLICA2_NAME"] %>
    username: <%= ENV["DATABASE_REPLICA2_USERNAME"] %>
    password: <%= ENV["DATABASE_REPLICA2_PASSWORD"] %>
    host: <%= ENV["DATABASE_REPLICA2_HOST"] %>

In the new database DSL that will land once PR #38721 is merged, there are configurations with multiple replicas on a primary database.

I know the connects_to only supports one reader ATM (e.g.: connects_to database: { writer: :primary, reader: :primary_replica }).

Is there a way pass in multiple readers for a primary database and let AR randomly choose on which replica it connects for READs instead of relying on a RDS load balancer? (e.g.: connects_to database: { writer: :primary, reader: [:primary_replica_one, :primary_replica_two] })

I’m fairly new to the multi DB world, so please forgive me if I didn’t articulate everything correctly. I’m here to learn and get better.

1 Like
2

You want to use the connects_to with the shards key rather than with the database key. That’s documented here Multiple Databases with Active Record — Ruby on Rails Guides

2 Likes
3

Hey @eileencodes,

From my understanding, the shards can only be used when doing horizontal sharding, but we only have one primary with two replicas at the moment.

Would we able to keep this architecture and, following the Multiple Databases with Active Record — Ruby on Rails Guides example, do something like this:

connects_to shards: {
    default: { writer: :primary, reader: :primary_replica },
    shard_one: { writer: :primary_shard_one, reader: :primary_shard_one_replica }
  }

Where the default and shard_one have the same primary DB in our config/database.yml for production (e.g.: my_primary_database)? Or will this work only if we split our primary DB and create the actual shard?

4

You can make a third role for the other replica but you’ll need to implement your own automatic functionality for it as Rails doesn’t have a concept of multiple readers by default. You also should be using writing/reading not writer/reader if you want any of the default functionality like preventing writes or the auto switcher to work.

connects_to database: { writing: :primary, reading: :primary_replica, reading_two: :primary_replica_two }
1 Like
5

@raulp a year later, how did your switch from Makara to the native Rails multiple DB support go? We’re considering moving to an AWS RDS multi-zone cluster with a read replica, so this is very interesting for us.

6

Hey @happycappie. Unfortunately, we did not make the switch because we felt the change was not worth it in terms of how much time&money we had to invest in writing our own replica switcher vs. the actual gain. We’re sticking with Makara for now. It serves us well.

1 Like
7

I have the same issue as @raulp with 1 writing primary and 2 reading replicas, which seems like a pretty common setup. I was excited to try Rails’ multiple database supports until I read the docs and realized it couldn’t work for us either.

I don’t think many people can use the current Rails multi DB support out of the box except a small niche that only have 1 writing primary and 1 reading replica.

Would be nice if Rails could support multiple reading replicas. Even better if it could do automatic role switching not only for web requests, but for SQL requests as well (e.g. SELECTs go to reading, INSERTs go to primary). Supporting this would involve basically re-implementing a lot of what Makara provides (like their proxy code), but would make Rails more batteries included for more people.

For now we keep using Makara, as I suspect most people in our case do. I am wondering if we are going against “the Rails way”. Maybe this is implicitly telling us that we should only have 1 primary and 1 replica, and that replica could be load-balanced outside of Rails with some ops work.

3 Likes