The build failed.
CHANGES
... etc.
Am I overreacting or is anyone else a tiny bit annoyed by the size of these emails? If you're mobile with a GSM card out of UTMS range , you have around 4-8kB/s, and 70kB per mail is a noticeable wait.
The summary at the top is fine, but I propose a link to the build details. If someone is interested in the gory details, they can open the link in a browser.
Forget about me if this is too much of a hassle.
Cheers.
I’m sorry. Here’s some options:
-
Filter these mails so you never see them (should be easy with Gmail)
-
Make it so. CruiseControl.rb is open source, and may even have this feature already, and all the CI config is in the rails repo. However, I think some people may like seeing the error output inline.
-
Help fix the build
– Chad
1. Best option, since even if the build is broken, bandwidth won't let me do much, anyway.
2. Maybe worth a look at least, but more importantly like you mentioned - some people may prefer having the errors inlined.
3. As in 1, doing a git-pull with a sloppy connection isn't a good
idea, and the problem will be fixed by the time I even finish the
fetch. And I don't even consider myself competent enough to even do
something about the broken build within a reasonable amount of time.
Unless in the unlikely event that all the Rails developers go on a
month-long holiday
Thanks for the reply and suggestions. Sorry for bringing this up.
If there are three commits—A, B and C, where A breaks the build and B,C are unrelated—I’m wondering why should CI send out emails for errors in B and C when it detected them in A.
In other words, CI should never re-send the same failures: only if they’ve changed. A trivial implementation of this would just be by comparing the standard output. Thoughts?
3. Help fix the build
If there are three commits—A, B and C, where A breaks the build and B,C are unrelated—I'm wondering why should CI send out emails for errors in B and C when it detected them in A. In other words, CI should never re-send the same failures: only if they've changed. A trivial implementation of this would just be by comparing the standard output. Thoughts?
"Don't forget the build is broken", thus, spam you with emails until someone drags his ass over and fixes the build.
Also, how do you *really* detect it's unrelated? Commit B might use code changed by A, which broke the build, so you don't really know if it doesn't fail.
Cheers -foca
The problem with the "spam emails" approach is that people will treat them as spam. I don't know what the answer is, but I wouldn't be surprised if a lot of the build mails to this group are going right to people's spam folders or being otherwise automatically disposed of.
Mike
I tend to print them out, beat them with a stick before I set them on
fire, unfortunately gmail was unable to automate this process for me.
Seriously though, I think you're right Mike. Actually, I can assure
you I know of at least a few people who have unsubscribed from the
list because of this. And as some find them annoying, plus it probably
doesn't do any good, why not just send them to a separate list where
people who actually _do_ want them can get them?
Eloy
I tend to print them out, beat them with a stick before I set them on fire, unfortunately gmail was unable to automate this process for me.
Seriously though, I think you're right Mike. Actually, I can assure you I know of at least a few people who have unsubscribed from the list because of this. And as some find them annoying, plus it probably doesn't do any good, why not just send them to a separate list where people who actually _do_ want them can get them?
+1 to that
+1
I tend to print them out, beat them with a stick before I set them on fire, unfortunately gmail was unable to automate this process for me.
Seriously though, I think you're right Mike. Actually, I can assure you I know of at least a few people who have unsubscribed from the list because of this. And as some find them annoying, plus it probably doesn't do any good, why not just send them to a separate list where people who actually _do_ want them can get them?
+1 to that
Eloy
"Don't forget the build is broken", thus, spam you with emails until someone drags his ass over and fixes the build
The problem with the "spam emails" approach is that people will treat them as spam. I don't know what the answer is, but I wouldn't be surprised if a lot of the build mails to this group are going right to people's spam folders or being otherwise automatically disposed of.
Isn't that why planecrashes happen?
I tend to print them out, beat them with a stick before I set them on fire, unfortunately gmail was unable to automate this process for me.
Seriously though, I think you're right Mike. Actually, I can assure you I know of at least a few people who have unsubscribed from the list because of this. And as some find them annoying, plus it
probably doesn't do any good, why not just send them to a separate list where people who actually _do_ want them can get them?
Actually, this *is* the right list. The motivation stirred up by the
series of CI emails either motivates you to help fix the problem or to
unsubscribe. That sounds about right. If seeing the Rails build broken
is too much of a burden, then perhaps being on the rails-core list
isn't the right place for you.
-Rob
Dude,
This is a list for discussions, and I'm hear to see if people might
have problems with code that I wrote for Rails, not for fun.
Please take your patronizing comments to the playground, you have
added 0.0 to this discussion.
Eloy
Argh, obviously I meant "here".
I agree with Rob that this list is the right place for it. It's easy
for individuals to "opt out" by setting up a mail filter. Takes 2
seconds.
- Ken
Hey Ken,
Sure it's 2 seconds of work to setup a mail filter, but so is signing
up for a specialized mailing list. So that doesn't make it a good
reason.
I wouldn't be bothered by these CI emails if we'd see one every now
and then. But nowadays, it seems like every other build is broken.
That's the real problem here.
Are people no longer running the test suite before committing? I can
see how you could get the occasional failure for db drivers other than
the one you use, in fact that's how Rails edge used to be. But it
hasn't been like that for a long time now.
Ask yourself, for whom is this info interesting? Not for me, I see
those failures when I run the tests of Rails. Especially before I
commit. So I surely hope that it's not being meant as a tool for devs
to no longer run the full suite yourself, because that's not what it's
there for plus it generates noise. If you want that noise, go ahead, I
just kindly ask the maintainers to take it to a specific list, as many
other projects do.
Cheers, Eloy
We use hudson, and we're very happy with it. It can be configured to only email to the person who broke the build.
Aside from that, the annoying part is indeed that the build is broken every hour.
Thanks for the feedback, everyone. FWIW, this is not new, build failures used to go to this list [1]
However, not fixing the build promptly is a problem, and one of the reasons I wanted to reinstate notifications to this list.
On a large/distributed project like Rails - especially one where many people run the master branch live in their projects - I believe there are exactly three appropriate responses to a broken build, in order of decreasing desirability:
1. Fix the failing tests, ASAP 2. Roll back the change which broke the tests, ASAP 3. Comment/disable the failing tests, ASAP
Any other response to a broken build is wrong and lazy, and leads to broken windows [2]. See my longer rant here: [3]
Now, it is FINE for the build to go red once in a while, especially for an open source project like Rails. It is not reasonable to expect the core team to run the full suite before every checkin, on every supported database and Ruby interpreter. That would be a waste of their valuable time and effort, and is why we set up CI to do it automatically.
However, LEAVING the build red is NOT fine. Spamming the list and annoying people is just a symptom of the real problem, which is not promptly resolving a broken build. Fixing the symptom by hiding the problem (sending failure emails to a separate list) is counterproductive, and only exacerbates the broken-window effect.
Anyway, I've disabled the notifications for now, and the core team has the ability to disable them anytime they want as well [4]. I didn't disable them because people complained, but because we just got a new suite of servers donated to set up an awesome, multi-branch, multi-interpreter Rails CI environment (thanks EngineYard!), and I have to complete that migration/setup.
[1] http://groups.google.com/group/rubyonrails-core/browse_thread/thread/2b610e77506812c0# [2] Pragmatic Bookshelf: By Developers, For Developers [3] http://pivots.pivotallabs.com/users/chad/blog/articles/484-how-you-can-learn-to-stop-worrying-and-love-continous-integration [4] http://github.com/rails/rails/blob/master/ci/cruise_config.rb#L4
The problem is that, unless I'm mistaken, the majority of the people on this list don't have commit rights, so we're unable to do any of these things ASAP. The value of having the notifications sent out to non committers is quite limited, which is why it would be nice to be able to opt out.
Thanks for the feedback, everyone. FWIW, this is not new, build failures used to go to this list [1]
How about a two stage commit. If CI repo works, then commit to main repo. If not, mail the commiter that the commit failed, while the whole world still has a working build. This is to prevent "broken windows" and the commiter usually has the most information about what could be broken and how to fix, anyway. Or get people pull always via a tag LAST_WORKING_BUILD, but that will be less obvious for many.
However, not fixing the build promptly is a problem, and one of the reasons I wanted to reinstate notifications to this list.
Originally, I was only noting the size. I prefer to read what people write than what machines spit out.
However, LEAVING the build red is NOT fine. Spamming the list and annoying people is just a symptom of the real problem, which is not promptly resolving a broken build. Fixing the symptom by hiding the problem (sending failure emails to a separate list) is counterproductive, and only exacerbates the broken-window effect.
I agree. In this case there are actually two broken widows - the commit and what people pull when they want the latest version. For me, the large-volume machine-generated mail became a third broken window in the way I follow what is going on in rubyonrails-core and how I personally use my mail reader.
Anyway, I've disabled the notifications for now, and the core team has the ability to disable them anytime they want as well [4]. I didn't disable them because people complained, but because we just got a new suite of servers donated to set up an awesome, multi-branch, multi-interpreter Rails CI environment (thanks EngineYard!), and I have to complete that migration/setup.
The best thing, as pointed out already, would be to prevent these things from ever happening.
That aside, I was toying of an idea of a tool like CI, but one that would run continuous integration tests to find the latest set of working gems using or being used by rails (plugins, libraries, etc). Something that could show, for example, if the latest commit to rails or rspec would break the other. An early warning for the commiter about which plugins will be broken and who to mail about what was changed.
Any ideas about this? Something already like this exists?
Cheers.
Perhaps a timeout within which the build should be fixed? I mean it doesn't take hours for CI to detect the problem. And then mail if fix doesn't come in time.
I don't believe committers will want to create and subscribe to a separate list. And this is *their* mailing list. Then again, the rest of us may want to know if we should "pull" or not.
