Below you'll find my proposal, please contact me if you have any
questions or suggestions:
My proposal is to improve the current developer tools available in Rails to :
1. Deliver more information to the user about his current Rails dev setup.
2. Collect these results to create reports on the different versions
and environments of Rails and
upload them to a central server containing all this information. Think
of "isitruby1.9" but for the
Rails project (not for individual Rails apps).
In the past several people have started projects to keep CC.rb
instances of Rails running to keep
developers notified of the changes to the project, but as far as I
know there is not a "canonical"
Rails CI to provide this information.
Chad Woolley has been working on an EC2 backed tool to provide this
service to the community
(http://github.com/thewoolleyman/railsci) but his current scope goes
up to providing a tool we can install
on a pristine server to see what the current build status of Rails is.
Although this is immensely
helpful by itself, I think we can gather more information than the
test results to see "How's Rails
My idea is to implement two tools that provide this information:
1. Improve the current Rails tools to get information about the
current version builds on developers
machines. This would include test results, performance information
(running times, memory usage,
GC-info, etc). Right now there are a couple of rake tasks that provide
this information to the user
when he runs it on his own applications (rake test:profile, rake
test:benchmark) but as far as I
know there are no tools to do this specifically on the Rails code
base. We'd have to write them
basing our work on the existing tools.
A simple patch to correct typos in the documentation or a refactoring
effort of a couple of methods might not
need to use these tools, but certain things like testing different
database connectors on different
Ruby implementations (REE vs. JRuby vs. MRI) could greatly benefit
from having access to this
information. On one hand the developers can find the slow points in
the system and try to fix them,
and in the other they'll be able to say "X implementation is probably
better suited to do Y or Z
tasks. When a user provides a bug report of Ruby 1.8.7 on OS X running
too slow we could just check
the results on our machines and see if it's a global issue or if it's
a local problem.
Once we have this information on the developer machines we could push
it to a centralized
repository. We could see how Rails behaves on different platforms and
environments. This is especially useful when doing changes to the low
level stuff (the first example
I could think of is playing with the send_file method) since these
kind of changes are really
dependant on the platform/web servers/OS used.
With this online tool we could see how a change affected the Rails
performance in general, and as an
added bonus we will also have access to the actual test results. This
is also good since we can
trivially see how the code changes in different environment:
Say that Arel decides to move the database connector code to a
DataObject backend. I write
this code, test it on my local machine and it works just fine, but
I don't have access to a
Windows system to try these tests on. I could just send a link to
the ML asking for someone to
please run a rake task or two and it will automatically test
everything, compile the results and
upload them to the website. There I would be able to see how
everything went and if there was a
problem I'll have instant access to that information.
I currently don't know how far Chad plans to take his project, but
from what I understand his idea
is to setup several EC2 images with different OS to try the code under
all the supported
platforms helping himself with RVM. If it works like this we'll be
able to integrate these results
into a "master" copy of the results (stuff we got from "official"
servers running clean versions of
Rails) so we have a baseline to compare against. If he's not planning
to support all the OS
combinations then we would still be able to get the results on all the
platforms (provided there are
enough users on those systems willing to run these tests).
I'd love to hear any feedback from you and your ideas on the project!