Hi Yehuda and Josh,
Thanks for all the time you're putting into ActionPack at the moment, it's really great. I'm sorry to say that I've noticed that there are a lot of changes in code and lots of stuff is moved around but I see relatively few changes to tests.
I'm a bit worried by this because I'm afraid this is going to cause a lot of bugs. Is there a reason why you're committing without tests?
Manfred
Hi,
It's probably no surprise that I agree with this email. What however
comes as a surprise to me is that nobody responded yet.
Are there no other people on this list that have concerns/opinions
about this issue?
Eloy
Well, I'm guessing the thinking is that the current tests should cover most of the changes. The problem with that is that a lot of the current tests are tied to implementation specifics which are obviously in a state of flux at the moment so (re)writing a whole bunch of tests is probably a bit premature so I'm willing to cut them so slack at the moment.
What we could do with is a decent set of specs that cover what's going to be the public API separate from the current test suite which is tightly coupled to the current codebase. We could also do with a test application for integration testing.
Andrew White
Well, I'm guessing the thinking is that the current tests should cover most of the changes.
Sure, but I see new code without tests in the commits.
The problem with that is that a lot of the current tests are tied to implementation specifics which are obviously in a state of flux at the moment so (re)writing a whole bunch of tests is probably a bit premature so I'm willing to cut them so slack at the moment.
That's why I said 'changes to tests', you would expect application specific tests to be disabled or removed if they start failing (which they obviously aren't because lots of them fail right now). I would expect a rewrite to happen in a separate branch and be merged once it's done, in which case it would contain tests. I had the impression that this was what Yehuda was doing, but maybe I'm wrong.
Even though I can't imagine doing a large refactor without backing it up with tests, I wouldn't mind their approach too much if they would let us know their plans. I obviously can't merge the Rails master branch with my own work right now because that would introduce too many test failures.
What we could do with is a decent set of specs that cover what's going to be the public API separate from the current test suite which is tightly coupled to the current codebase. We could also do with a test application for integration testing.
That would be great, but that would also be supplemental to unit tests.
Manfred
Sure, but I see new code without tests in the commits.
On a cursory look I don't see anything that intends to change existing behaviour - can you point something out.
I would expect a rewrite to happen in a separate branch and be merged once it's done, in which case it would contain tests. I had the impression that this was what Yehuda was doing, but maybe I'm wrong.
Probably the reason for merging back in now before refactoring has been completed is that it becomes harder and harder to merge without conflicts becoming very difficult to resolve - e.g. the work done on improving performance in development mode in the run-up to 2.3 has been ignored for the moment.
Even though I can't imagine doing a large refactor without backing it up with tests, I wouldn't mind their approach too much if they would let us know their plans. I obviously can't merge the Rails master branch with my own work right now because that would introduce too many test failures.
I agree it's all a little bit mysterious at the moment - I'm putting off submitting anything substantial to Lighthouse because I don't want to waste time on something that's being refactored away. It'd be nice to know who is exactly working on what - for example is your work on refactoring AR scopes an 'official' project or just something you've taken upon yourself? It's been mentioned in a few LH tickets which I might have taken a stab at. I'm assuming that there's a core team IRC/Campfire chat area where these things are being discussed.
What we could do with is a decent set of specs that cover what's going to be the public API separate from the current test suite which is tightly coupled to the current codebase. We could also do with a test application for integration testing.
That would be great, but that would also be supplemental to unit tests.
Obviously. 
Andrew White
It'd be nice to know who is exactly working on what - for example is your work on refactoring AR scopes an 'official' project or just something you've taken upon yourself?
I sort of agreed with Pratik that I would work on it. I don't know how official that makes it (:
I'm assuming that there's a core team IRC/ Campfire chat area where these things are being discussed.
I think something like that exists, but I've never seen it with my own eyes.
Manfred
My apologies for the delay in responding. I'm a bit new to all of this. I'll try to explain what's going on exactly.
There are basically two kinds of changes that Carl and I have been making:
1) Refactoring of existing code. For the most part, the existing tests have held up reasonably well. You won't see too many changes to tests because we're trying to ensure test passage as a baseline for determining whether we have broken things. Rewriting the tests and also refactoring code at the same time is a recipe for breakage, so we're trying, as much as possible, to leave the existing tests as canonical "rails is working" tests. Most of the implementation- specific stuff is in helpers, so there are usually very small changes in helpers that clean up changes due to refactor. That said, we're definitely involved in a long-term effort to make the tests less implementation-specific.
2) New implementations. Two examples of this are the new_base work in progress (experimental work on a more modular implementation of AC::Base) and Carl's router (http://github.com/carllerche/rack-router/ tree/master). Those efforts involve rather serious new speccing efforts. For instance, you can see the new_base specs at http://github.com/rails/rails/tree/256b0ee8e3c1610967dfc89f864e24b98ed3c236/actionpack/test/new_base. Much of the behavior tested there (e.g. "rendering a normal template with full path with layout => true") was either not tested or not tested systematically. These tests are both ported from old tests in a less implementation-specific manner, or verified against a new Rails 2.3 application. We're obviously not done yet (by any stretch), but if you go through the new_base specs, I think you'll be pleased with our thoroughness.
I'd be happy to answer any other questions you might have.
-- Yehuda
In addition to Yehuda's comments, I would add that there are several major initiatives planned for the Rails 3 timeframe:
(1) Existing tests should be upgraded to be more thorough as the code is touched. There are various spots in the Rails 2.x codebase where tests could be improved, either because the state of testing has advanced, or because a patchwork of tests can be unified, or for other reasons.
(2) There's been some discussion of having a thorough integration testing script that can exercise the process of building and using a Rails application. This is at the "blue sky" level so far, but there's some work from Sam Ruby with automating the Depot application from AWDWR that points the way. Ultimately, this should give us a good working smoke test for Rails 3.
(3) The continuous integration servers that are used to test Rails itself are getting overhauled, with an eye towards getting us faster results on a wider variety of configurations.
So, the people working on Rails 3 are *definitely* interested in good testing, on a variety of levels. I firmly believe that as Rails 3 moves forward, we'll all be able to look back and say that it's the best-tested version of Rails ever.
Mike
Nice, thanks for the explanation. Hopefully now I won't have any sleepless nights anymore (:
Manfred
I'd like to see the CI build kept green through these efforts:
http://ci.rubyonrails.org/builds/rails
Is that feasible, or are things in transition and you are planning to getting back to green at some future point?
Thanks, -- Chad
Ah, it is green again. Nice
-- Chad
Keep up to date with Rails on Twitter and This Week in Rails
Policies: Conduct, License, Maintenance, Security, Trademarks