Rails 2.0: Database Adapters as Plugins or RubyGems?

In response to my uber patch for the OpenBase database adapter, DHH
mentioned that the intention for 2.0 was to pull most of the adapters
out into plugins so that development could continue outside of the
core. I think the end result is admirable - the core team doesn't
need to waste time applying patches for databases that they have no
means or inclination to install/use/test/etc. The core team should be
designing AR, and the adapter maintainers figure out what this means
for their specific database backend (understanding, of course, that AR
design can't occur in a bubble ... I don't mean to say that
implementation should not be a consideration in AR design decisions).
However, in thinking through the user experience I think there may be
an even better way.

If some adapters, say mysql and postgres, remain in core while the
rest become plugins, then we will create a discontinuity in the user
experience. For some databases, you need only to specify your adapter
in database.yml and you're good to go. For other databases, you'll
need to not only adjust database.yml, but also perform this extra step
of running script/plugin install ... Then there's the question of
matching adapter versions to versions of Rails.

But what if the adapters were bundled as RubyGems? After all,
installing the rails gem means installing the gems that comprise
rails. What if the gem for each of the adapters was listed as a
dependancy for rails? Then they would already be installed and
available to everybody using rails as a gem. When the database.yml
file is parsed, we can load the proper gem based on the adapter
setting. The adapters are not in the core, but they can all be
accessed without loading plugins. And, pardon me if my understanding
of RubyGems is demonstrated to be painfully lacking, but couldn't we
also say that a specific version of rails requires a certain version
of the adapter gem?

Of course, the elephant in the room is, what to do when rails is not
being run as a gem. How would this affect rails:freeze:edge?

Just my crazy idea for the day...
-Derrick Spell

But what if the adapters were bundled as RubyGems?

Nice idea, it seems to me that it goes in the same direction as the (proper) rails core wrt distribution…

And, pardon me if my understanding
of RubyGems is demonstrated to be painfully lacking, but couldn’t we
also say that a specific version of rails requires a certain version
of the adapter gem?

I think this is correct…

Of course, the elephant in the room is, what to do when rails is not

being run as a gem. How would this affect rails:freeze:edge?

What about using svn:externals or simply modify the rails:freeze:edge to pull the adapter code from some external repository?

But the real question is: would it be a good thing to make rails core depend on external libraries (not in the standard ruby distribution)? I’d dare to say the it is already this case as this happens with most of the libraries used by the database adapters.

From another point ov view: also today different database adapters depend on external libraries (typically gems) to handle the lower levels i.e. also for rails 1.2, with the oracle adapter in the core you still need to install the oci gem to be able to connect (with platform specific code), the same goes for the mysql adapter if you want to use the native libs, and so on for the others.

This make me think that in the end we may even let the adapters live in plugins and let the user specify them on application creation (in addition to being able to add them with the usual script/plugin install).

rails --with-adapter=oracle …

N.B. I’d make this command (and the adapter plugin install) check and eventually pull down not only the adapter plugin but also all the correct required libraries (gems and such). In the end the amount of work required to the user could be even less than today :slight_smile:

cheers,
Luca

Of course, the elephant in the room is, what to do when rails is not
being run as a gem. How would this affect rails:freeze:edge?

“Vendor everything.”
Simply unpack the adapter to yourapp/vendor directory. So you see, this does not represent a problem.

And, pardon me if my understanding of RubyGems is demonstrated to be painfully lacking, but couldn’t we also say that a specific version of rails requires a certain version of the adapter gem?

Gems can have dependencies targeting a specific version of a specific gem. While it doesn’t make sense for Rails to have dependencies for specific adapters, it makes sense for these adapter gems to have dependencies on a specific version of AR.

Gems can have dependencies targeting a specific version of a specific gem.
While it doesn't make sense for Rails to have dependencies for specific
adapters, it makes sense for these adapter gems to have dependencies on a
specific version of AR.

Yes, that's more correct. The adapters will depend on a certain version of AR.

-Derrick

From another point ov view: also today different database adapters depend on
external libraries (typically gems) to handle the lower levels i.e. also for
rails 1.2, with the oracle adapter *in the core* you still need to install
the oci gem to be able to connect (with platform specific code), the same
goes for the mysql adapter if you want to use the native libs, and so on for
the others.
This make me think that in the end we may even let the adapters live in
plugins and let the user specify them on application creation (in addition
to being able to add them with the usual script/plugin install).

# rails --with-adapter=oracle ...

The only difference I would say there is between having to load the
libraries and having to load the adapters as plugins is that you only
need to load the library once per machine, whereas you would need to
load the adapter as a plugin for every rails app. Having the
generator do some of this for you would be nice, but we will still
lose the ability to easily change databases by simply changing
database.yml and running a rake task. Now you will also have to make
sure you install the proper plugin when you change databases. We can
argue that this doesn't happen often, but still it is a convenience
that will be lost.

What about those developers who like to use SQLite for the development
environment, but use something else for testing and production? I
think it comes down to convience, and it's not inconvenient to have to
load something once during installation, but it might be considered
inconvenient to have to load it everytime you start a project or
change databases. Perhaps the answer would be to have rails load the
adapters as gems, but not have the gems installed along with rails.
You can load the proper adapter gem at the same time you load the
corresponding library gem, and then it's available to whatever project
may need or want it.

-Derrick

In my opinion, those extra adapter should be distributed as gems. Installing them as plugins should be discouraged because you have to repeat it for every app. If you already chose to go with some non-mainstream db backend, the chances are you’ll be using the same for all your applications anyway.

And if you’re unable to install the gem at your web host, you can always unpack it in “vendor/” or simply copy the gem file in “vendor/gems” after this gets in:
http://dev.rubyonrails.org/ticket/8511

So, when will the adapters be extracted? Are there known maintainers for every one? Are they going to be in external repositories?

yes this makes sense, also something i didn’t think about: probably the database adapter depends more in the machine environment than on the single application (as in: my development machine has the sqlite db and required gem, while my production machine ahs the postgresql database and gem installed)

Luca

In my opinion, those extra adapter should be distributed as gems. Installing
them as plugins should be discouraged because you have to repeat it for
every app. If you already chose to go with some non-mainstream db backend,
the chances are you'll be using the same for all your applications anyway.

And if you're unable to install the gem at your web host, you can always
unpack it in "vendor/" or simply copy the gem file in "vendor/gems" after
this gets in: http://dev.rubyonrails.org/ticket/8511

So, when will the adapters be extracted? Are there known maintainers for
every one? Are they going to be in external repositories?

We have an active oracle maintainer, but the other adapters tend to
float around from time to time. They'll be extracted before 2.0, and
we'll need to run a maintainer hunt around that time.

As for internal vs external repositories, I'd say that external
repositories are the best option, but if it's a deal breaker for
someone I'm sure we could carve out a repository here for them.

Whether we choose gems or plugins for the adapters, there's some work
needed to make sure that it works smoothly. At present the adapters
are required very early in the initialization process, so hopefully
someone who's maintaining / creating an adapter can help us get the
required changes into AR.

As for internal vs external repositories, I'd say that external
repositories are the best option, but if it's a deal breaker for
someone I'm sure we could carve out a repository here for them.

Since I'm now also maintaining the Ruby/OpenBase library, I can move
the OB adapter into the rubyforge repos. Actually, that makes things
rather clean and tidy. Perhaps a similar arrangement could be made
for other adapters...

Whether we choose gems or plugins for the adapters, there's some work
needed to make sure that it works smoothly. At present the adapters
are required very early in the initialization process, so hopefully
someone who's maintaining / creating an adapter can help us get the
required changes into AR.

That sounds rather like a call to arms :wink: I've got a lot on my plate,
but as soon as I get a little time I'll take a look. Unless someone
else beats me to it, of course...

-Derrick Spell

That sounds rather like a call to arms :wink: I've got a lot on my plate,
but as soon as I get a little time I'll take a look. Unless someone
else beats me to it, of course...

That's exactly what it was! :slight_smile:

The jruby guys have a 'jdbc' adapter which may benefit from the same
work. Perhaps hit them up?

I've written a custom adapter and packaged it as a plugin before.
It's not open source, but it was fairly simple to get working:

Added to config/environment.rb:

config.load_paths += %W( #{RAILS_ROOT}/vendor/plugins/blah_adapter/lib )
config.connection_adapters = %w( blah )

Then I put the adapter file into my plugin:

vendor/plugins/blah_adapter/lib/active_record/connection_adapters/blah_adapter.rb

Hope that helps whoever decides to work on this.