plugin dependencies

Hello all,

The topic of plugin dependencies has come up before and it doesn't
seem to have been addressed by core or core doesn't seem to think it's
an issue. I've looked at the current edge code and don't see anything
new, so if I've missed something *please* let me know.

The following article makes mention of a require_plugin functionality...
http://www.pluginaweek.org/2006/11/05/plugin-dependencies-revisited/

and the referenced article seems to make the point that such a feature
may not be required...
http://weblog.techno-weenie.net/2006/10/31/plugin-dependencies

The reason I bring this up is that with many current core features
being moved to plugins, existing plugins that have been written to
improve functionality in some of those core areas may no longer work
since they will now be dependent on these "core plugins" being
installed... and even if they are installed, there's no guarantee that
the load order will be correct.

For example, I have written a plugin to improve the functionality of
the in_place_editor code. This code works just fine with the current
public version of rails. Now in edge this core code has been moved to
a plugin and my plugin will no longer work as expected since the core
plugin is now "required" by my plugin. This is fine (although a more
informational warning would be useful for the user) as the user simply
needs to install the "core plugin" as well. This is where problems
start to occur. Since rails loads plugins in alphabetical order by
default, if my plugin comes before the core plugin in name, my plugin
code is overridden.

With many rails features being deprecated and being moved to plugins,
this will cause many dependencies on any existing plugins. This can
be resolved by specifying the load order with the config.plugins
option, but a user shouldn't have to jump through those hoops to get
things working... at least not before a fair amount of possible
debugging since there may be no indication of the underlying problem
of an existing plugin not working (a user may believe it's their own
code causing problems). Another possible solution is making sure ones
own plugin is alpabetically higher than the one it depends on to force
the proper load order, but that would be rather nasty hack.

So I'd like to know what people think about having a "require_plugin"
type functionality in core? Is there a valid workaround that exists
already? If there isn't, do people even consider this a problem?

Thanks,
Andrew

You can actually control the order of plugin loading with
config.plugins array, but I'd guess that'd be PITA

I believe require_plugin ( I'd prefer just "plugin" ) would be a good
to have feature.

You can actually control the order of plugin loading with
config.plugins array, but I'd guess that'd be PITA

Eg if plugin_a, previously without dependencies starts depending on plugin_b (where say plugin_b is some bit of core that has recently been pushed to a plugin). You have to go and edit config.plugins of all the apps using plugin_a.

I believe require_plugin ( I'd prefer just "plugin" ) would be a good
to have feature.

I'd love that too. We've been using plugins to share code between our apps and this would really make life easier

Fred

I mentioned config.plugins in my OP...

"This can be resolved by specifying the load order with the
config.plugins option, but a user shouldn't have to jump through those
hoops to get things working..."

and yes, PITA is right.

"This can be resolved by specifying the load order with the
config.plugins option, but a user shouldn't have to jump through those
hoops to get things working..."

We're still pretty skeptical about plugin dependencies on the whole.
Plugins monkeypatching plugins seems like a bit of a recipe for
disaster. However we could do something easy like:

config.plugins = [:gems, :some_kind_of_thing, :all]

So you specify the key things at the beginning of the array, then just
use :all to mean 'then everything else in alphabetical order'.

Seems like that'll solve most of the more-common problems where a
specific plugin needs to be early on the load path.

"This can be resolved by specifying the load order with the
config.plugins option, but a user shouldn't have to jump through those
hoops to get things working..."

We're still pretty skeptical about plugin dependencies on the whole.
Plugins monkeypatching plugins seems like a bit of a recipe for
disaster. However we could do something easy like:

It seems to me that it's perfectly possible to depend on the presence of stuff without necessarily patching it (and to be fair a lot of the time you don't need access to your dependencies at plugin load times in which case it all just goes away (which has been the case when we've been using plugins internally to share code between apps) but I wouldn't want to rely on it always working out that easily).

Fred

It seems to me that it's perfectly possible to depend on the presence
of stuff without necessarily patching it (and to be fair a lot of the
time you don't need access to your dependencies at plugin load times
in which case it all just goes away (which has been the case when
we've been using plugins internally to share code between apps) but I
wouldn't want to rely on it always working out that easily).

If the plugins aren't dependent on each other at load time, the we
don't need something like require_plugin.

It seems to me that it's perfectly possible to depend on the presence
of stuff without necessarily patching it (and to be fair a lot of the
time you don't need access to your dependencies at plugin load times
in which case it all just goes away (which has been the case when
we've been using plugins internally to share code between apps) but I
wouldn't want to rely on it always working out that easily).

If the plugins aren't dependent on each other at load time, the we
don't need something like require_plugin.

Sure, and what we've done so far internally is twist things so that this isn't the case and we don't have to rely on load order. It would be nice not to have to perform those contorsions.

Fred

Sure, and what we've done so far internally is twist things so that
this isn't the case and we don't have to rely on load order. It would
be nice not to have to perform those contorsions.

So for those situations you could use config.plugins =
[:your_first_thing, :all] ? Or am I missing something.

Sure, and what we've done so far internally is twist things so that
this isn't the case and we don't have to rely on load order. It would
be nice not to have to perform those contorsions.

So for those situations you could use config.plugins =
[:your_first_thing, :all] ? Or am I missing something.

Yes you could, but that puts the onus on the containing application. I've got say 3 plugins that my 5 apps share. Then I add plugin X and then change plugin A to use something funky from plugin X. I don't what to encode this new information in 5 different environment.rb files. It would be nice for that knowledge to stay in plugin A. It's okish for internal stuff when the author of the plugin and the consumer of the plugin are the same person/team, but it's a bit nasty if you're releasing a plugin to a wider audience.

Fred

A very minor change that would help mitigate this type of issue is
separating the addition of plugins to the load path from the
evaulation of init.rb.

If every plugin's lib was in the load path *before* any plugin could
start playing around or monkeypatching, many situations where you
might want to explicitly ensure a plugin was loaded can be made to go
away.

Perhaps some mechanism of changing the order config.plugins from within a plugin would be simpler, e.g:

Three plugins A, B and C need to be loaded in the order C, A, B as B depends on A & C and A depends on C.

In a dependencies.rb within the plugin's top level directory:

   Plugin A: depends_on :plugin_c
   Plugin B: depends_on :plugin_a, :plugin_c
   Plugin C: <empty>

The plugin loader would start off with the alphabetical list and then remove and re-insert each plugin depending on it's dependencies. e.g:

   Initial: [:plugin_a, :plugin_b, :plugin_c]
   A/deps : [:plugin_b, :plugin_c, :plugin_a] # A re-inserted after C
   B/deps : [:plugin_c, :plugin_a, :plugin_b] # B re-inserted after A
   C/deps : [:plugin_c, :plugin_a, :plugin_b] # No changes

I'm sure there's some situation which wouldn't work but at least it help reduce the need to manually specify config.plugins.

Andrew White

   Initial: [:plugin_a, :plugin_b, :plugin_c]
   A/deps : [:plugin_b, :plugin_c, :plugin_a] # A re-inserted
after C
   B/deps : [:plugin_c, :plugin_a, :plugin_b] # B re-inserted
after A
   C/deps : [:plugin_c, :plugin_a, :plugin_b] # No changes

After looking at that, I don't think it's a good idea to support it at
all. Mainly because plugins have no cocept of versions like gems. And
because of distributed nature of plugins, it can break things very
easily. If plugins really have this level of dependancy, they're
better off duplication the functionality I guess.

I thought I'd offer my 2 cents on the subject. plugin_dependencies
has evolved since that article was written. I never use
require_plugin anymore and instead rely on simply calling require,
which allows the dependency to be loaded either as a plugin or as a
gem. As is, my plugin development practices allow me to release
plugins as both a Rails plugin and a Ruby gem.

Say we have 2 plugins, encrypted_strings and encrypted_attributes
(which depends on encrypted_strings). In this case, we do the
following:

encrypted_attributes/init.rb:
require 'encrypted_attributes'

encrypted_attributes/lib/encrypted_attributes.rb:
require 'encrypted_strings'
...do stuff...

With plugin_dependencies loaded, this will look for encrypted_strings
as either a vendor/plugin within the application or as a gem. This
lets me be flexible in case I want the same plugin/gem/whatever used
across multiple applications in a single environment. The basic idea
is that we're not specifically tying a library as a plugin using
require_plugin, but instead treating it the same as you do with Rails
either being loaded locally in vendor/ or as a gem.

With regard to partial loading, both myself and James Adam use the
following technique:

Rails::Initializer.run do |config|
  config.plugins = %w(
    plugins_plus
    something_else

Ignoring anything about requiring specific versions, couldn't this be
achieved by having all of the plugins added to the load path before
any init.rb files are evaluated (my suggestion above)? That way the
normal Ruby "require" would seem to provide everything that you
describe here.

That still means that plugins aren't really plug-in anymore. If I use a
plugin that needs acts_as_list at initialization, and that plugin happens
to be named "aardvark" instead of "zebra", I'll have to go manually add
acts_as_list to the beginning of config.plugins? Why not just manually
copy the plugin code into lib/ as well?

I understand the core team's traditional reluctance to add plugin
dependencies, for both complexity and YAGNI reasons. I remember a ticket a
while back where somebody said "But if I have 70 plugins, it becomes a
pain", and DHH rightly responded "You don't need 70 plugins".

But with the move towards pushing more and more core functionality into
plugins, it really should be revisited - if only to be re-argued in the
present tense. I'm working on a relatively small app on Edge that already
uses a dozen plugins, and we've just gotten started. I don't think that's
going to be uncommon.

Do you really want to go back to the days where we all had to roll our own
linked lists?

Indeed, I actually had proposed this way back when:
http://dev.rubyonrails.org/ticket/6418#comment:17. I'm not sure why I
came to the conclusion that this was already being done on edge. I
think this did result in some other issues that needed to be
considered, such as:
1. When a plugin is loaded, it's lib path always comes before other
plugins, therefore if I trying to require a file called "node", it'll
look in my plugin's lib path before looking elsewhere. This would not
be the case with the paths added before initialization.
2. Similarly, the order of the paths would not reflect the order in
which the plugins were loaded, resulting in potential issues with the
priority in which files are loaded
3. You would still need support for loading gems as plugins because
otherwise they would not go through the same motions as normal plugins
(such as evaluating the init.rb).
4. You'd need support for loading tasks from gems.
5. If you call require 'encrypted_strings', and encrypted_strings/lib/
encrypted_strings.rb gets loaded, then the plugin is still going to
get loaded again later on if it's in vendor/plugins. This again means
that the list of plugins loaded is out of order and would break
engines/plugins+.

That's all I can think of right now. I certainly agree that this
would be simpler, although I'm not sure if it brings up additional
issues that need to be addressed. Any technique that allows us to
describe dependencies in the plugins and not in the application's
configuration I support.

So let me get it straight.

I release Plugin A. Joe release Plugin B which uses "your"
functionality to depend on my Plugin A. One fine morning I get
enormous motiviation to improve performance of my Plugin. So I take a
day off at work and rewrite the entire damn thing and release it. Poor
Joe has no clue what I did. So every person installing his Plugin B,
will have broken functionality because noone has a goddman idea about
what I did with Plugin A last night.

Welcome to Stone age.

If I were you, I'd refrain from personal attacks.

That "70 plugins" somebody was me and I guess I was an outlier back
then :slight_smile: Probably still am :wink: