Redshift adapter code works in Rails 6.1, broken(?) in 7.2.2.1 (activerecord7-redshift-adapter)

Rails upgrades Upgrade from 6.1 to 7.0
1

Greetings

I will limit this first post to fewer details; will provide just enough to give an idea about the problem, in case someone knows and is willing to help. Will provide more details if I get replies…

In Rails 6.1 I was able to query AWS Redshft cluster using ActiveRecord::Base, with adapter: “redshift” in the configuration parameters:

class myModel <  ActiveRecord::Base 
end

rs_params = {
            adapter: "redshift",
            host: "<hostid>.redshift.amazonaws.com",
            port: "<port>",
            database: "<dbname",
            username: "<username>",
            password: "<password>",
            pool: 3,
            timeout: 10
          }

then…

myModel.establish_connection(rs_params)

That was it!

I think it was putting to use ‘activerecord6-redshift-adapter’ –

----- Now in Rails 7.2.2.1 everything appears broken. With activerecord7-redshift-adapter

For starters, I get:

/active_record/connection_adapters/redshift/schema_statements.rb:4:in ‘module:Redshift’: uninitialized constant ActiveRecord::ConnectionAdapters::AbstractAdapter::SchemaCreation (NameError)

The reference path to the SchemaCreation appears wrong. I am finding “SchemaCreation” sitting in the above incorrectly stated grandparent relationsihp… So I do a patchy thing in my script, moving “SchemaCreation” up to subclass of ConnectionAdapters (already feeling jittery – this isn’t supposed to need patching… I may be simply entangling myself further). The “patch”:

class ActiveRecord::ConnectionAdapters::AbstractAdapter::SchemaCreation < ActiveRecord::ConnectionAdapters::SchemaCreation
end

That breaks my (explicit) require of the adapter component ( require ‘active_record/connection_adapters/redshift_adapter.rb’ ), until I move the require AFTER the above “patch”

OK. The above gets me past the “SchemaCreation” not found error – it is now being found – or rather its duplicate class created to match the reference.

Now it starts telling me the adapter “redshift” doesn’t exit. I rummage through activerecord7-redshift-adapter-1.1.0/lib/active_record/connection_adapters/redshift_adapter.rb and am seeing the adapter name is now capitalized … (??? wtf):

 class RedshiftAdapter < AbstractAdapter
   ADAPTER_NAME = 'Redshift'.freeze
  ....

OK. I change ‘redshift’ to ‘Redshift’. I now get an error saying the ‘Redshift’ adapter is not registered…

I dig in the code and add registration manually:

ActiveRecord::ConnectionAdapters.register(
  'Redshift',
  'ActiveRecord::ConnectionAdapters::RedshiftAdapter',
  '/active_record/connection_adapters/redshift_adapter.rb'
)

That gets me past the registration error.

Now I get

wrong number of arguments (given 1, expected 4)

/active_record/connection_adapters/redshift_adapter.rb:189:in ‘initialize’

Yeah… I see this in the above file:



      def initialize(connection, logger, connection_parameters, config)
        super(connection, logger, config)

        @visitor = Arel::Visitors::PostgreSQL.new self
        @prepared_statements = false

        @connection_parameters = connection_parameters

top of the stack trace:

active_record/connection_adapters/redshift_adapter.rb:189:in 'initialize'
active_record/database_configurations/database_config.rb:26:in 'Class#new'
active_record/database_configurations/database_config.rb:26:in 'ActiveRecord::DatabaseConfigurations::DatabaseConfig#new_connection'
active_record/connection_adapters/abstract/connection_pool.rb:883:in 'ActiveRecord::ConnectionAdapters::ConnectionPool#new_connection'

I don’t know where to get the needed arguments

connection, logger, connection_parameters, config

and why they are not being inserted in the call … seems like a mismatch of functions… a mess.

I paused my hackery here, to post

What happened between the versions ? seems like we went from simple implementation to complexity, broken at that… or am I missing some new component that has been introduced? Can’t find documentation on this.

thanks.

2

Update - new observation:

I see that activerecord7-redshift-adapter-1.1.0/lib/active_record/connection_adapters/redshift_adapter.rb provides this method:

ActiveRecord::ConnectionHandling.redshift_connection(config),

which just takes a hash config as argument, which resembles what I had in the 6.1 version, with a single config as argument to .establish_connection(config). It instantiates and returns “ConnectionAdapters::RedshiftAdapter.new(nil, logger, conn_params, config)” – the same method that fails with the wrong number of arguments (1 not 4), posted earlier, above.

So, I try to do call this method, as a probe-test –

myConnection = ActiveRecord::ConnectionHandling.redshift_connection(rs_params)

This errors out with this:

undefined method ‘register_class_with_limit’ for an instance of ActiveRecord::ConnectionAdapters::RedshiftAdapter

I am finding that the method does exist in:

/active_record/connection_adapters/abstract_adapter.rb

( ActiveRecord::ConnectionAdapters::AbstractAdapter )

And the RedshiftAdapter should also have it, because it is defined as inheriting from the above:

class RedshiftAdapter < AbstractAdapter

but it appears not to inherit the method…

Either I am missing a simple / correct way to invoke the Redshift adapter, or this is terribly broken

(please help!)

3

Because of the single hash argment it takes, reasonable to assume that the provided custom Redshift method “redshift_connection(config)” in ActiveRecord::ConnectionHandling should be inserted in the called stack; it would then, in its turn, properly call the constructor for class “ConnectionAdapters::RedshiftAdapter”, with the 4 arguments that we see it provides, and that i am getting errorred on – but it looks like that method is being bypassed - (possibly because of my “patching” – aliasing classes in the hierarchy to skip a level (?))

Further more, the method ActiveRecord::ConnectionAdapters::RedshiftAdapter does not get constructed by anything else, it appears (and it most certainly should! right?)

I added an “initialize” (it is missing one, originally), with pry:

      def initialize()

        puts "When are we being initialized? "
        binding.pry

      end

And Pry never fires at the above point… suggesting the class instance never gets constructed, for the “establish_connection” call

4

Hello, all

I think it would be useful to add my latest new observations (new for me)

Some version difference specifics that look impacting here

  1. the /active_record/database_configurations/database_config.rb component of ActiveRecord has this big rewrite between 6.1 and 7.2:

The 6.1 version routes connection requests in method “adapter_method”, which builds the adapter-type specific method name, based on the provided adapter literal, like so:

 def adapter_method
   "#{adapter}_connection"
 end

In the 7.2 version, the old “adapter_method” is gone, replaced with “adapter_class” method, which sets up a whole distinct class for each adapter type (it seems), like so:

 def adapter_class
   @adapter_class ||= ActiveRecord::ConnectionAdapters.resolve(adapter)
 end
  1. The adapter gem itself has undergone significant revision from 6.1 to 7

(if anyone is interested I can post the diff here)

A couple of interesting things I see (as possible clues to the fix), in active_record/connection_adapters/redshift_adapter.rb

A) The 6.1 version has this adapter registration in the file itself:

ActiveRecord::Tasks::DatabaseTasks.register_task(/redshift/, "ActiveRecord::Tasks::PostgreSQLDatabaseTasks")

It has been removed in the 7 version, without replacement, apparently

B) Both 6.1 and 7 versions have a “redshift_connection” method

    def redshift_connection(config)
      conn_params = config.symbolize_keys

      conn_params.delete_if { |_, v| v.nil? }
      ....

Interestingly, the “#{adapter}_connection” construct (in /active_record/database_configurations/database_config.rb, see above) seems to align with this method, but it has been removed in 7.2, but the literal method with the name “redshift_connection” is still there. In other words, in 6.1 there is something dynamically building a call handle to the “redshift_connection” method, but in version 7 nothing appears to build this type-specific (and class-specific) method name.

5

perhaps my final appeal to the community; a reminder:

if anyone feels like tackling this for sport – I think this error could be debugged without having an actual connection to AWS redshift resource

If one gets to an error message about invalid credentials or endpoint, one likely has reached a fix – that would mean the code has contacted AWS

thanks.

6

The fork gem nicknamed ‘pennylane’ seems to work flawlessly: