Rails, Activism and Lack of API

Merb was always putting out this line about how Rails had no public
API. I found it laughable at first because http://api.rubyonrails.org/
seemed to obviously refute this. However, recently I've noticed this
same line still being trotted about, by "Rails" "activists" no less.
Can someone give me an honest answer why this what seems to me to be a
baldface lie is being promoted from within our own ranks now?

Thanks.

RSL

As I told you on twitter, here is the official word from David
himself: http://weblog.rubyonrails.org/2008/12/23/merb-gets-merged-into-rails-3

"Rigorous API: Too many plugins break when Rails is updated because
it’s not clear where they can safely hook into the internals and when
they’re monkeypatching and should expect things to break. The Merb
guys committed to a public API with tests to ensure that it wouldn’t
break. They’ll bring over that line of thinking and give Rails 3 a
tested and documented API for extensions that won’t break willy-nilly
with upgrades."

The rest of the details is being worked on.

- Matt

Not to be flip, but as I replied on Twitter as well
http://twitter.com/rsl/status/1860374823 ... How does one rectify the
fact of the existence of Rails API http://api.rubyonrails.org/ and the
large test suite that Rails has with the comment that Rails _will_ not
_does_ have these things? Restating your position doesn't alleviate
the need to back it up. Perhaps it's just [as people are saying] some
kind of language/semantic difference but IMO it's something that needs
to be set straight. Especially when being said by those who present
themselves as the face of Rails. Until we can clear up all this [and
start using the right terms when "activisting" ] it seems to me to
boil down to the same old FUD we had before where Merb is obviously
superior to Rails and is coming to our rescue instead of a syncretism
of two views into an altogether better one.

RSL

Hold your horses. Rails having no public API is a mis-statement of a
very real shortcoming. As you point out, a public Application PI is
there indeed. What's missing is a defined interface for people extending
or otherwise hacking Rails.

The usual approach is just to do what currently works. Consequently,
code like that is bound to break on updates to the Rails code base. A
defined interface could future-proof such code.

It's unfortunate that the term API is used in this context as that's not
what the problem is about. An SPI (Service PI) it is neither. I have no
suggestion for a better term.

Michael

I agree that Rails has made many changes to it’s public API and how we could all benefit from more stability [as well as the irony that in order to get this fabled stable API, we’re going to break all of the previous one]. It’s the language of all this that bugs me and I feel needs to change in order to communicate accurately and honestly the situation instead of relying on the old responses and accusations. Were this not being used in the public forum by “activists”, I wouldn’t even raise the question.

RSL

There are two ways of thinking about what’s an API:

  1. bunch of methods declared public which have some RDoc documentation so they appear on api.rubyonrails.org;
  2. an interface with strong conventions and multiple layers where each layer corresponds to one field of usage (Rails users, framework plugins/extensions, and component internals) and where there’s some guarantee that some API will exist at least until version X.Y.
    The problem with Rails versions up to now is there was no guarantee. Versions 2.0 -> 2.1 -> 2.2 -> 2.3 have given and taken methods away mercilessly, and people who wrote ActionPack hacks or ActiveRecord associations plugins felt that hard. Yehuda Katz once said that Rails thought in the old days that they don’t need no extensions API, everyone can monkeypatch everything. Time has shown that this is not true; you cannot inject a feature with a monkeypatch if a target method is gone/renamed/moved in the next release.

So this is what I think activists are talking about when they say “public API”: guarantee. Some level of security.

I agree that Rails has made many changes to it's public API and how
we could all benefit from more stability [as well as the irony that
in order to get this fabled stable API, we're going to break all of
the previous one].

I'm not sure you've understood what I wrote. Do you realize that there
is a difference between interfaces targetted at application developers
as opposed to interfaces targetted at framework extension programmers?

It's the language of all this that bugs me and I
feel needs to change in order to communicate accurately and honestly
the situation instead of relying on the old responses and
accusations. Were this not being used in the public forum by
"activists", I wouldn't even raise the question.

As far as language is concerned, my immediate impression was that you're
attacking(!) this issue in a rather unfortunate way. Too much
aggression, too little understanding. If you feel piqued by my wording,
you'll appreciate what I mean.

Michael

Merb was always putting out this line about how Rails had no public
API. I found it laughable at first because
http://api.rubyonrails.org/ seemed to obviously refute this. However,
recently I've noticed this same line still being trotted about, by
"Rails" "activists" no less. Can someone give me an honest answer why
this what seems to me to be a baldface lie is being promoted from
within our own ranks now?

Hold your horses. Rails having no public API is a mis-statement of a
very real shortcoming. As you point out, a public Application PI is
there indeed. What's missing is a defined interface for people extending
or otherwise hacking Rails.

The usual approach is just to do what currently works. Consequently,
code like that is bound to break on updates to the Rails code base. A
defined interface could future-proof such code.

It's unfortunate that the term API is used in this context as that's not
what the problem is about. An SPI (Service PI) it is neither. I have no
suggestion for a better term.

EPI = Extension Programming Interface?
PPI (Plugin...)

Right. And Public API != "interfaces targetted at framework extension
programmers". That's plugin API.

I completely understand the difference between a public interface and an internal one meant for extending the framework. However, the language [whether intentional or not] implies that Rails has neither which is not the case. My complaint is with the clumsy, casual language [formerly employed as FUD by the Merb camp] still being promulgated under the guise of Rails “activism”. If one is not talking about the public API, one should not use the term public API cf: http://twitter.com/merbist/status/1838558766. If one is talking about a specific definition of API, one would do well to clarify this specific definition rather than using the plain term as if everyone automatically understands that by “API” one means something else.

On the plus side, I don’t feel piqued by your reply at all. In fact I thank you for the free ad hominem, strawman, and intellectual condescension you’ve provided. I’ll keep it around 'til I need it.

RSL

Here are some ideas for parts of the future public API:

  • monitoring & profiling: hooks for NewRelic/TuneUp/Scout instrumentation;
  • ActiveRecord fulltext search & indexing: for Sphinx and other plugins. the core implementation, however, would be just using common RDBMS features such as “WHERE name LIKE ‘%foo%’”;
  • authentication & authorization: for restful_authentication, authlogic, etc.;
  • testing API & helpers: so that rspec-rails, webrat and other integrations wouldn’t break between releases.

Here are some current examples of what I consider a public API of Rails:

ActionController::Base.param_parsers[Mime::XML] = Proc.new { |data| … }

ActionView::Template.register_template_handler(‘foo’, FooHandler)

I18n.backend = MyBackend.new

All of these are used to inject features into the framework by implementing some interface, not monkeypatching. This is good.

What really saddens me is that now all focus is turned towards middleware and that they’ve become almost the preferred way to inject features into the framework. Tons of Rails features were already extracted. The whole stack, including the main app, is then synchronized through a single object: “env”. Is the public API going to become string keys on a big fat hash object?

Here are some ideas for parts of the future public API:

monitoring & profiling: hooks for NewRelic/TuneUp/Scout instrumentation;

We have a 'probes' based summer of code project for exactly this kinda thing.

ActiveRecord fulltext search & indexing: for Sphinx and other plugins. the
core implementation, however, would be just using common RDBMS features such
as "WHERE name LIKE '%foo%'";

I'm not 100% sold that there's something abstractable here. The
semantic and syntactic differences are much more drastically different
between sphinx, solr, mysql fulltext and tsearch2 than between
postgresql and mysql's INSERT or SELECT statements.

authentication & authorization: for restful_authentication, authlogic, etc.;

This is something I'm *strongly* opposed to including in the core of
rails until we have an absolutely bullet proof and popular solution
from various plugins. yehuda and carl have spoken about porting the
general design of merb-auth into something for rails 3, and we can
probably start there.

testing API & helpers: so that rspec-rails, webrat and other integrations
wouldn't break between releases.

You can see the start of this for integration and functional testing
with the (planned) move to rack-test for the implementation. That
should give everyone a common block on which to build.

Here are some current examples of what I consider a public API of Rails:
ActionController::Base.param_parsers[Mime::XML] = Proc.new { |data| ... }
ActionView::Template.register_template_handler('foo', FooHandler)
I18n.backend = MyBackend.new
All of these are used to inject features into the framework by implementing
some interface, not monkeypatching. This is good.

Right, to say we don't have an extension API is pure FUD, what we do
have is a TINY extension API that needs to get bigger. The way it'll
get bigger is for people who maintain plugins to suggest what APIs
they'll need, and help us get them included and stabilised.

The alternative is for us to invent extension points from our ivory
tower (or rimu single story home). The odds of us getting it 'mostly
right' are pretty good, but with feedback from plugin authors we'll
probably get it completely right.

What really saddens me is that now all focus is turned towards middleware
and that they've become almost the preferred way to inject features into the
framework. Tons of Rails features were already extracted. The whole stack,
including the main app, is then synchronized through a single object: "env".
Is the public API going to become string keys on a big fat hash object?

There are a few cases where we've probably gone overboard (I'm still
not sold on the exceptions changes) however we can always suck them
back in before we cut a 3.0 release. However at a fundamental level
if we can provide useful middlewares for sinatra, ramaze and other
guys to use, why not?

Looking at merb-auth right now makes me feel that’s exactly what I expect from a framework to have.

About middlewares: while I agree with more than half of the extractions (I’m all for sharing between frameworks), like you I’m not sold on some of the stuff. But what worries me is: is there a standard on Rack env key names? Are they documented? Are middlewares versioned independently?

I’ve written middleware that manipulates request parameters before it reaches the router, so all of these questions gone through my head while I was researching all the env values used.