Rails 2.0: Database Adapters as Plugins or RubyGems?

In response to my uber patch for the OpenBase database adapter, DHH mentioned that the intention for 2.0 was to pull most of the adapters out into plugins so that development could continue outside of the core. I think the end result is admirable - the core team doesn't need to waste time applying patches for databases that they have no means or inclination to install/use/test/etc. The core team should be designing AR, and the adapter maintainers figure out what this means for their specific database backend (understanding, of course, that AR design can't occur in a bubble ... I don't mean to say that implementation should not be a consideration in AR design decisions). However, in thinking through the user experience I think there may be an even better way.

If some adapters, say mysql and postgres, remain in core while the rest become plugins, then we will create a discontinuity in the user experience. For some databases, you need only to specify your adapter in database.yml and you're good to go. For other databases, you'll need to not only adjust database.yml, but also perform this extra step of running script/plugin install ... Then there's the question of matching adapter versions to versions of Rails.

But what if the adapters were bundled as RubyGems? After all, installing the rails gem means installing the gems that comprise rails. What if the gem for each of the adapters was listed as a dependancy for rails? Then they would already be installed and available to everybody using rails as a gem. When the database.yml file is parsed, we can load the proper gem based on the adapter setting. The adapters are not in the core, but they can all be accessed without loading plugins. And, pardon me if my understanding of RubyGems is demonstrated to be painfully lacking, but couldn't we also say that a specific version of rails requires a certain version of the adapter gem?

Of course, the elephant in the room is, what to do when rails is not being run as a gem. How would this affect rails:freeze:edge?

Just my crazy idea for the day... -Derrick Spell

But what if the adapters were bundled as RubyGems?

Nice idea, it seems to me that it goes in the same direction as the (proper) rails core wrt distribution…

And, pardon me if my understanding of RubyGems is demonstrated to be painfully lacking, but couldn’t we also say that a specific version of rails requires a certain version of the adapter gem?

I think this is correct…

Of course, the elephant in the room is, what to do when rails is not

being run as a gem. How would this affect rails:freeze:edge?

What about using svn:externals or simply modify the rails:freeze:edge to pull the adapter code from some external repository?

But the real question is: would it be a good thing to make rails core depend on external libraries (not in the standard ruby distribution)? I’d dare to say the it is already this case as this happens with most of the libraries used by the database adapters.

From another point ov view: also today different database adapters depend on external libraries (typically gems) to handle the lower levels i.e. also for rails 1.2, with the oracle adapter in the core you still need to install the oci gem to be able to connect (with platform specific code), the same goes for the mysql adapter if you want to use the native libs, and so on for the others.

This make me think that in the end we may even let the adapters live in plugins and let the user specify them on application creation (in addition to being able to add them with the usual script/plugin install).

rails --with-adapter=oracle …

N.B. I’d make this command (and the adapter plugin install) check and eventually pull down not only the adapter plugin but also all the correct required libraries (gems and such). In the end the amount of work required to the user could be even less than today :slight_smile:

cheers, Luca

Of course, the elephant in the room is, what to do when rails is not being run as a gem. How would this affect rails:freeze:edge?

“Vendor everything.” Simply unpack the adapter to yourapp/vendor directory. So you see, this does not represent a problem.

And, pardon me if my understanding of RubyGems is demonstrated to be painfully lacking, but couldn’t we also say that a specific version of rails requires a certain version of the adapter gem?

Gems can have dependencies targeting a specific version of a specific gem. While it doesn’t make sense for Rails to have dependencies for specific adapters, it makes sense for these adapter gems to have dependencies on a specific version of AR.

Gems can have dependencies targeting a specific version of a specific gem. While it doesn't make sense for Rails to have dependencies for specific adapters, it makes sense for these adapter gems to have dependencies on a specific version of AR.

Yes, that's more correct. The adapters will depend on a certain version of AR.

-Derrick

From another point ov view: also today different database adapters depend on external libraries (typically gems) to handle the lower levels i.e. also for rails 1.2, with the oracle adapter *in the core* you still need to install the oci gem to be able to connect (with platform specific code), the same goes for the mysql adapter if you want to use the native libs, and so on for the others. This make me think that in the end we may even let the adapters live in plugins and let the user specify them on application creation (in addition to being able to add them with the usual script/plugin install).

# rails --with-adapter=oracle ...

The only difference I would say there is between having to load the libraries and having to load the adapters as plugins is that you only need to load the library once per machine, whereas you would need to load the adapter as a plugin for every rails app. Having the generator do some of this for you would be nice, but we will still lose the ability to easily change databases by simply changing database.yml and running a rake task. Now you will also have to make sure you install the proper plugin when you change databases. We can argue that this doesn't happen often, but still it is a convenience that will be lost.

What about those developers who like to use SQLite for the development environment, but use something else for testing and production? I think it comes down to convience, and it's not inconvenient to have to load something once during installation, but it might be considered inconvenient to have to load it everytime you start a project or change databases. Perhaps the answer would be to have rails load the adapters as gems, but not have the gems installed along with rails. You can load the proper adapter gem at the same time you load the corresponding library gem, and then it's available to whatever project may need or want it.

-Derrick

In my opinion, those extra adapter should be distributed as gems. Installing them as plugins should be discouraged because you have to repeat it for every app. If you already chose to go with some non-mainstream db backend, the chances are you’ll be using the same for all your applications anyway.

And if you’re unable to install the gem at your web host, you can always unpack it in “vendor/” or simply copy the gem file in “vendor/gems” after this gets in: http://dev.rubyonrails.org/ticket/8511

So, when will the adapters be extracted? Are there known maintainers for every one? Are they going to be in external repositories?

yes this makes sense, also something i didn’t think about: probably the database adapter depends more in the machine environment than on the single application (as in: my development machine has the sqlite db and required gem, while my production machine ahs the postgresql database and gem installed)

Luca

In my opinion, those extra adapter should be distributed as gems. Installing them as plugins should be discouraged because you have to repeat it for every app. If you already chose to go with some non-mainstream db backend, the chances are you'll be using the same for all your applications anyway.

And if you're unable to install the gem at your web host, you can always unpack it in "vendor/" or simply copy the gem file in "vendor/gems" after this gets in: http://dev.rubyonrails.org/ticket/8511

So, when will the adapters be extracted? Are there known maintainers for every one? Are they going to be in external repositories?

We have an active oracle maintainer, but the other adapters tend to float around from time to time. They'll be extracted before 2.0, and we'll need to run a maintainer hunt around that time.

As for internal vs external repositories, I'd say that external repositories are the best option, but if it's a deal breaker for someone I'm sure we could carve out a repository here for them.

Whether we choose gems or plugins for the adapters, there's some work needed to make sure that it works smoothly. At present the adapters are required very early in the initialization process, so hopefully someone who's maintaining / creating an adapter can help us get the required changes into AR.

As for internal vs external repositories, I'd say that external repositories are the best option, but if it's a deal breaker for someone I'm sure we could carve out a repository here for them.

Since I'm now also maintaining the Ruby/OpenBase library, I can move the OB adapter into the rubyforge repos. Actually, that makes things rather clean and tidy. Perhaps a similar arrangement could be made for other adapters...

Whether we choose gems or plugins for the adapters, there's some work needed to make sure that it works smoothly. At present the adapters are required very early in the initialization process, so hopefully someone who's maintaining / creating an adapter can help us get the required changes into AR.

That sounds rather like a call to arms :wink: I've got a lot on my plate, but as soon as I get a little time I'll take a look. Unless someone else beats me to it, of course...

-Derrick Spell

That sounds rather like a call to arms :wink: I've got a lot on my plate, but as soon as I get a little time I'll take a look. Unless someone else beats me to it, of course...

That's exactly what it was! :slight_smile:

The jruby guys have a 'jdbc' adapter which may benefit from the same work. Perhaps hit them up?

I've written a custom adapter and packaged it as a plugin before. It's not open source, but it was fairly simple to get working:

Added to config/environment.rb:

config.load_paths += %W( #{RAILS_ROOT}/vendor/plugins/blah_adapter/lib ) config.connection_adapters = %w( blah )

Then I put the adapter file into my plugin:

vendor/plugins/blah_adapter/lib/active_record/connection_adapters/blah_adapter.rb

Hope that helps whoever decides to work on this.