taking the Rails API to a next level

As I mentioned earlier today, there some ideas to take the API to a
next level. I think Ruby on Rails is a fucking awesome software, and
it should have a fucking awesome API.

Yehuda is working on a different way to organize content. Rizwan Reza
is exploring the idea of supporting user comments, there was also a
guy in Baltimore that volunteered to have a look into that.

But there are also other ideas that I think are interesting at least
to be investigated, if any of you want to work on any of them that
would be totally great!

First of all an axiom:

    It is OK that we build our own doc tool ad-hoc for Rails,
    be creative!

Four existing ideas, they may conflict with each other but we will
sort it out if the case arrives:

== User-Centered API

Most of the modularity in the Rails code base is there by design. But
an end-user does not really care whether AC::Base has been broken in
these many modules. If you go to AC::Base you should have all the
public interface right there.

Accomplishing this is difficult with parsing, since there are
inherited hooks going on, autoloads, code like

    MODULES.each do |mod|
      include mod
    end

etc. The wild idea is to go and load recursively all the Rails lib
directories (except perhaps for a few exceptions with side-effects).
With all Rails libs executed, all hooks executed, etc. introspect in
memory, and build meaningful docs. You can delegate docstring
extraction to an existing parser.

== Linking Tests to the API

The API should be written in English, but sometimes it would be handy
to be able to consult right there the tests for a given method. You
know, a block with tests that is hidden by default, akin to the
current code block.

Here you need to investigate a way to link tests and methods, and be
able to insert them in the API.

== Testing Rails Guides

Rails Guides are really really great, but they need to be maintained.
If they had code coverage that would be included with the Rails test
suite and people would be aware of content being outdated by a patch.
That would help a lot, and hey, having tested documentation isn't an
exciting idea by itself? Yehuda, José, and Sam Ruby have done stuff in
that line for books (with different approximations). If you'd like to
take a stab at this it would be a good idea to have at least a look at
their solutions.

== Awesome Theme

This depends heavily on the rest, but anyway it is very important. For
the API to be fucking awesome, it has to *look* fucking awesome. The
idea here is to have a gorgeous theme, possibly based on the color
palette and overall style of Rails Guides, and Rails Contributors for
consistency's sake.

== Testing Rails Guides

Rails Guides are really really great, but they need to be maintained.

If they had code coverage that would be included with the Rails test

suite and people would be aware of content being outdated by a patch.

That would help a lot, and hey, having tested documentation isn’t an

exciting idea by itself? Yehuda, José, and Sam Ruby have done stuff in

that line for books (with different approximations). If you’d like to

take a stab at this it would be a good idea to have at least a look at

their solutions.

I think I ran out of “fucking awesome” ideas / “win” for this week already, so excuse me if this sounds outlandish. I love the guides and I think that code coverage would be great for them. I don’t think this is as possible in their current (Textile) format as it would be in say a DocBook (XML)-esque format.

[pause to allow you to shudder]

Yeah, all about the large sweeping changes this morning, but hear me out on this one too.

It doesn’t have to be exactly DocBook (the format I’m using for my book right now certainly isn’t!), just like it. What I’m imagining is that you have a git repository of code that you write the documentation against. When you make a commit, you’ll be able to reference that file in the format like this:

[line 8]

[line 9]

[line 10]

Where in this example, ref is a Git ref, meaning branch, tag, commit, whatever. You know this stuff already. Yeah, go ahead and point the ref at master. Point it at 2-3-stable. Point it at whatever you’re writing the documentation for. Here we may wish to use a commit sha as this part of the documentation will be based on a particular stage of our application, or we may wish to reference a particular branch of Rails. Up to you.

The file points to a path at some pre-configured “super-path” which in this imaginary case is the root of the repository. Then the start and end lines in this file at this ref should be the lines inside the programlisting, and this is where the automated testing comes in.

The automated testing tool parses the format, finds the programlisting and looks up application.html.erb at this ref and checks lines 8-10 are exactly what’s in the programlisting. If they’re not, then you’ll get a “failure”. This way, the automated testing ensures that 1) if someone makes a change in the code the test breaks and 2) if the code in the example changes, the test breaks. Same method as standard TDD.

The next Problem To Solve would be ensuring output is what you say it is. I imagine the way would be similar to the file tag:

<![CDATA[rake test && something_else && whatever]] 10 examples, 0 failures, 3 pending Again, the imagined application will checkout the whatever-we're-documenting to this sha, and then run the command and ensure the output is exactly the text in the documentation. If we only want to make sure the output contains then I suggest setting a "contains" attribute to true on the output tag. I just don't think this is what Textile was made for and I think if we continue using it and trying to implement a testing solution for it we get ye olde "round peg, square hole" scenario. XML is easy to parse (NOKOGIRI FOR LIFE *double tap chest into sideways peace sign*) into a HTML format, which is what we're doing with Textile anyway. So what do you think?

I'm not convinced that this would work well for testing on the scale required in the Rails source, but we use an adaptation of Python's doctest over on Hobo:

http://github.com/tablatom/rubydoctest

--Matt Jones

Yes, the rubydoctest in Hobo is very impressive IMHO.

- Owen Dall

Very rough (lunch time) implementation of the imagined idea here: http://github.com/radar/needs-a-name. Works fine for start -> end, but not for refs. Maybe another lunch break later, or if anyone else is interested on working on this the offer is there.

etc. The wild idea is to go and load recursively all the Rails lib
directories (except perhaps for a few exceptions with side-effects).
With all Rails libs executed, all hooks executed, etc. introspect in
memory, and build meaningful docs. You can delegate docstring
extraction to an existing parser.

My 2cents: someone actually already implemented "the wild idea" here
http://github.com/dolzenko/reflexive
and here
http://github.com/dolzenko/method_extensions (using Ripper parser).

The problems I've encountered:

class_eval still screws the things up, also pretty often there are
layers of behavior-extending modules included in AR::Base before
you get to the actual behavior-defining method (which will have
the docstring you're interested in). This means that using this
approach you will have to specify where to look up particular
methods (like some methods of AR::Base should be looked up
on AR::Relation, etc.)

I think I ran out of "fucking awesome" ideas / "win" for this week already, so excuse me if this sounds outlandish. I love the guides and I think that code coverage would be great for them. I don't think this is *as* possible in their current (Textile) format as it would be in say a DocBook (XML)-esque format.

[pause to allow you to shudder]

shudder.

Yeah, all about the large sweeping changes this morning, but hear me out on this one too.

It doesn't have to be *exactly* DocBook (the format I'm using for my book right now certainly isn't!), just like it. What I'm imagining is that you have a git repository of code that you write the documentation against. When you make a commit, you'll be able to reference that file in the format like this:

<example>
  <programlisting file="app/views/layouts/application.html.erb" start="8" end="10" at="[ref]">
    [line 8]
    [line 9]
    [line 10]
  </programlisting>
</example>

[...]

The next Problem To Solve would be ensuring output is what you say it is. I imagine the way would be similar to the file tag:

<screen ref="[sha]">
  <command>
    <![CDATA[rake test && something_else && whatever]]
  </command>
  <output>
    10 examples, 0 failures, 3 pending
  </output>
</screen>

[...]

So what do you think?

Don't know about you, but I would not like to write documentation in XML. Though this looks a lot like YARD would be an option. YARD is super extensible and already supports stuff like @example. Also, at the last Ruby User Group Berlin meeting Yehuda mentioned YARD could be an option for Rails.

Konstantin

etc. The wild idea is to go and load recursively all the Rails lib
directories (except perhaps for a few exceptions with side-effects).
With all Rails libs executed, all hooks executed, etc. introspect in
memory, and build meaningful docs. You can delegate docstring
extraction to an existing parser.

My 2cents: someone actually already implemented "the wild idea" here
http://github.com/dolzenko/reflexive
and here
http://github.com/dolzenko/method_extensions (using Ripper parser).

That's fantastic! So the planets are aligned :).

The problems I've encountered:

class_eval still screws the things up, also pretty often there are
layers of behavior-extending modules included in AR::Base before
you get to the actual behavior-defining method (which will have
the docstring you're interested in). This means that using this
approach you will have to specify where to look up particular
methods (like some methods of AR::Base should be looked up
on AR::Relation, etc.)

Sure, I think there will be still unsolvable things for an automated
tool, perhaps we have methods dynamically generated inside a method
definition, the callbacks stuff is particularly dynamic for example...
It won't be perfect, but I guess overall it will significantly much
better.

Excellent Evgeniy, I need to look into your projects in detail, but
from a first impression I think they should be definitely included in
the Rails tool-chain, or inspire a custom solution.

What I am envisioning at this moment is:

1. There's a script that uses your code to build a static[*] website
based on runtime introspection.

2. There's some new macro in unit tests or something to link them to
methods, I don't know, something like

    covers 'Some::Module#some_method'

and method_extensions, or an adaptation of it for Rails, would provide
a #tests accessor.

3. There's a gorgeous template the script in #1 uses to combine and
present everything.

-- fxn

[*] Unless we introduce user comments, I think it is better to keep it
static and simple. User comments are hard I think, that'd be for
another thread. A search box would also be included but that can also
be static.

PS: Ryan, need to think about your proposal, will write back later.

Excellent Evgeniy, I need to look into your projects in detail, but
from a first impression I think they should be definitely included in
the Rails tool-chain, or inspire a custom solution.

What I am envisioning at this moment is:

1. There's a script that uses your code to build a static[*] website
based on runtime introspection.

Thanks! Another thing that bothered me is how to make sure that
every part of framework is loaded (in this case this would be required
before you can go ahead and scan classes for methods they define).
I'm thinking about the autoload const_missing stuff,
on_load hooks hooks etc.

[*] Unless we introduce user comments, I think it is better to keep it
static and simple. User comments are hard I think, that'd be for
another thread. A search box would also be included but that can also
be static.

AFAIK Disqus is OK with static sites?
2 cents again: at the moment I recall 2 documentation sites with
user comments enabled (PHP and new jQuery site) and
in both cases it never seemed very useful to me.