Proposal: ActiveRecord transaction middleware

rubyonrails-core
,
1

There are some cases where you’d like to set certain local configuration (e.g. SET LOCAL for Postgres) around transactions within ActiveRecord in circumstances where session/global configuration is not possible (e.g. transaction-pooled pgBouncer). Something akin to Rack middleware would be helpful for achieving this: you’d be able to configure the global middleware used around all transactions (e.g. set a statement_timeout to 10s but override it in some specific cases (e.g. for a heavy migration, you could set it to 1m).

3 Likes
2

Seems interesting. How would you configure the middleware and how would you use it? Can you provide just like a pseudo sample of the API you want? It doesn’t need to be perfect, we can iterate on it.

Thanks.

3

I don’t think it needs to be more complex than accepting something that responds to call and yields to the block that is provided. It would likely make sense to take some configuration in an initializer or config or whatever:

ActiveRecord::Base.transaction_middleware << (proc do |_transaction, connection, &blk|
  connection.execute("SET LOCAL statement_timeout TO '5s'")
  yield
  # Don't need to reset anything as `SET LOCAL` statement only lasts until end of transaction
end)
  • I’m sure you could also make the ordering api more sophisticated (append, insert_after , etc.)
  • You could allow inline additions like
ActiveRecord::TransactionMiddleware.with(my_middleware) do
  Book.transaction { ... } # The middleware would take effect here
end
  • You could also allow for inline disabling of middleware if you had an identifier or a reference (e.g. Book.transaction(without_middleware: [:statement_timeout]) { ... } ) or you could do a block like the previous item
4

I personally like the simplicity of just appending a proc to a list. Are you thinking we just put that in an initializer?

Would these txn middleware apply to all connections? Like we don’t need to split them for like read replicas for example?

This would override all txn middleware, or add more middleware?

So that we don’t have to track key value pairs in Rails, maybe we pass a context object to “transaction”, then it can pass that object to the blocks, and the blocks can skip themselves (or do whatever they want) based on the data in the context object. Wdyt?

Also do you know if there are gems or apps out there doing this already? I’ve not needed this feature before, but I definitely see the utility. It’d be nice to see IRL code trying to accomplish this (including working around the shortcomings in Rails apis).

2 Likes
5

I personally like the simplicity of just appending a proc to a list. Are you thinking we just put that in an initializer?

Yes!

This would override all txn middleware, or add more middleware?

I think this would add more middleware–if you wanted to remove middleware, you could do conditionals in the middleware itself like with the context bit you were talking about. Perhaps there would be an api for adding to the “front” and adding to the “back” of the array which would cover all cases I think

Would these txn middleware apply to all connections? Like we don’t need to split them for like read replicas for example?

Hmm I had not thought about replicas as we don’t use them. Perhaps there’d be a separate array for those? I don’t have a strong opinion there.

So that we don’t have to track key value pairs in Rails, maybe we pass a context object to “transaction”, then it can pass that object to the blocks, and the blocks can skip themselves (or do whatever they want) based on the data in the context object. Wdyt?

Yea I think that makes sense. And things can modify the context obj (akin to the env in Rack) to interact with one another

Also do you know if there are gems or apps out there doing this already? I’ve not needed this feature before, but I definitely see the utility. It’d be nice to see IRL code trying to accomplish this (including working around the shortcomings in Rails apis).

I’m not familiar with any and wasn’t able to find anything from googling. It seems a bit risky to take on as a gem as you are messing with a pretty core API IMO.

1 Like
7

Slightly related, there recently was a PR to support parallel query execution in Postgres using SET as well.