Limiting the number of open database connections in migrations

I work on a Rails app with a multi-tenancy setup. When a new customer is added we dynamically create a new database for them. In our migrations we override ActiveRecord::DatabaseConfigurations#configs_for to load the configurations for all customer databases, so that we can run bundle exec rails db:migrate and it “just works”.

However, the environment where we run migrations (GCP Cloud Run jobs + Cloud SQL) has an artificial limit of 100 database connections at a time, which we cannot currently work around. I’m wondering, is there a way I can reliably limit the total number of database connections that the migration will use at any one time? Limiting connection pool size doesn’t work, as far as I can tell, because we get a separate connection pool for each customer database, meaning that even with a limit of 1 we still open Customer.count connections over the total lifetime of the migration, (which as far as I can tell are never closed).

I played around with using a custom connection pool which immediately closes the connection when it’s checked in, but it doesn’t seem to work as I’d hoped. There’s also the possibility of forking migrations across multiple jobs, though if possible I’d prefer to avoid that as we need to deal with partial failures (e.g. by rolling back successful shards when another fails).

Would appreciate any suggestions or help folk can provide! Thanks! :pray: