While diagnosing this problem with Jelmer Vernooij (Bazaar), it seems that there is a certain "misconfiguration" on the part of Rails Subversion server hosted on Joyent.
If this is a Joyent-side issue, would someone with authority of the Rails SVN server please contact Joyent about this problem so we can resolve it?
Thank you.
Rails Subversion repository])
Hi Hendy,
There aren't any configurations to do with SVN other than turning it on. What's happening is no different than standard HTTP errors when you're trying to push or pull too much stuff. No different then trying to put a string that's over 1024 bytes into IE, or trying to pass a string that's larger to 4096 bytes to Apache (so something like passing 200 facebook "friends" IDs in a string.)
Yes, I see the same thing with pulling up a PATH on the URL of the svn repository
Jasons-15er% svn log -v http://svn.rubyonrails.org/rails trunk svn: REPORT request failed on '/rails/!svn/bc/7514' svn: REPORT of '/rails/!svn/bc/7514': Response exceeded maximum number of header fields. (http://svn.rubyonrails.org)
The error isn't from subversion it's from the Neon library that subversion uses for http and webdav access to stuff.
In ne_request.c, the max header is 8192 bytes
#define MAX_HEADER_LEN (8192)
/* Read response headers. Returns NE_* code, sets session error and * closes connection on error. */ static int read_response_headers(ne_request *req) { char hdr[MAX_HEADER_LEN]; int ret, count = 0;
while ((ret = read_message_header(req, hdr, sizeof hdr)) == NE_RETRY && ++count < MAX_HEADER_FIELDS) { char *pnt; unsigned int hash = 0;
/* Strip any trailing whitespace */ pnt = hdr + strlen(hdr) - 1; while (pnt > hdr && (*pnt == ' ' || *pnt == '\t')) *pnt-- = '\0';
/* Convert the header name to lower case and hash it. */ for (pnt = hdr; (*pnt != '\0' && *pnt != ':' && *pnt != ' ' && *pnt != '\t'); pnt++) { *pnt = ne_tolower(*pnt); hash = HH_ITERATE(hash,*pnt); }
/* Skip over any whitespace before the colon. */ while (*pnt == ' ' || *pnt == '\t') *pnt++ = '\0';
/* ignore header lines which lack a ':'. */ if (*pnt != ':') continue;
/* NUL-terminate at the colon (when no whitespace before) */ *pnt++ = '\0';
/* Skip any whitespace after the colon... */ while (*pnt == ' ' || *pnt == '\t') pnt++;
/* pnt now points to the header value. */ NE_DEBUG(NE_DBG_HTTP, "Header Name: [%s], Value: [%s]\n", hdr, pnt); add_response_header(req, hash, hdr, pnt); }
if (count == MAX_HEADER_FIELDS) ret = aborted( req, _("Response exceeded maximum number of header fields"), 0);
return ret; }
Jasons-15er% svn log -v http://svn.rubyonrails.org/rails trunk svn: REPORT request failed on '/rails/!svn/bc/7514' svn: REPORT of '/rails/!svn/bc/7514': Response exceeded maximum number of header fields. (http://svn.rubyonrails.org) Jasons-15er% svn log -v http://svn-commit.rubyonrails.org/rails trunk
wc -l
46380
So let's take a look and see
Nonverbose is fine
Jasons-15er% svn log http://svn.rubyonrails.org/rails trunk | wc -l 23132
Oddly enough, verbose on the entire repos URL is also fine (leave off trunk) ... try and figure out that one
Jasons-15er% svn log -v http://svn.rubyonrails.org/rails | wc -l 66627
Verbose on a specific version is fine
Jasons-15er% svn log -v -r 7000 http://svn.rubyonrails.org/rails trunk
jason wrote:
Hi Hendy,
There aren't any configurations to do with SVN other than turning it on. What's happening is no different than standard HTTP errors when you're trying to push or pull too much stuff. No different then trying to put a string that's over 1024 bytes into IE, or trying to pass a string that's larger to 4096 bytes to Apache (so something like passing 200 facebook "friends" IDs in a string.)
Yes, I see the same thing with pulling up a PATH on the URL of the svn repository
Jasons-15er% svn log -v http://svn.rubyonrails.org/rails trunk svn: REPORT request failed on '/rails/!svn/bc/7514' svn: REPORT of '/rails/!svn/bc/7514': Response exceeded maximum number of header fields. (http://svn.rubyonrails.org)
The error isn't from subversion it's from the Neon library that subversion uses for http and webdav access to stuff.
In ne_request.c, the max header is 8192 bytes
#define MAX_HEADER_LEN (8192)
/* Read response headers. Returns NE_* code, sets session error and * closes connection on error. */ static int read_response_headers(ne_request *req) { char hdr[MAX_HEADER_LEN]; int ret, count = 0;
while ((ret = read_message_header(req, hdr, sizeof hdr)) == NE_RETRY && ++count < MAX_HEADER_FIELDS) { char *pnt; unsigned int hash = 0;
/* Strip any trailing whitespace */ pnt = hdr + strlen(hdr) - 1; while (pnt > hdr && (*pnt == ' ' || *pnt == '\t')) *pnt-- = '\0';
/* Convert the header name to lower case and hash it. */ for (pnt = hdr; (*pnt != '\0' && *pnt != ':' && *pnt != ' ' && *pnt != '\t'); pnt++) { *pnt = ne_tolower(*pnt); hash = HH_ITERATE(hash,*pnt); }
/* Skip over any whitespace before the colon. */ while (*pnt == ' ' || *pnt == '\t') *pnt++ = '\0';
/* ignore header lines which lack a ':'. */ if (*pnt != ':') continue;
/* NUL-terminate at the colon (when no whitespace before) */ *pnt++ = '\0';
/* Skip any whitespace after the colon... */ while (*pnt == ' ' || *pnt == '\t') pnt++;
/* pnt now points to the header value. */ NE_DEBUG(NE_DBG_HTTP, "Header Name: [%s], Value: [%s]\n", hdr, pnt); add_response_header(req, hash, hdr, pnt); }
if (count == MAX_HEADER_FIELDS) ret = aborted( req, _("Response exceeded maximum number of header fields"), 0);
return ret; }
Jasons-15er% svn log -v http://svn.rubyonrails.org/rails trunk svn: REPORT request failed on '/rails/!svn/bc/7514' svn: REPORT of '/rails/!svn/bc/7514': Response exceeded maximum number of header fields. (http://svn.rubyonrails.org) Jasons-15er% svn log -v http://svn-commit.rubyonrails.org/rails trunk > wc -l 46380
So let's take a look and see
Nonverbose is fine
Jasons-15er% svn log http://svn.rubyonrails.org/rails trunk | wc -l 23132
Oddly enough, verbose on the entire repos URL is also fine (leave off trunk) ... try and figure out that one
Jasons-15er% svn log -v http://svn.rubyonrails.org/rails | wc -l 66627
Verbose on a specific version is fine
Jasons-15er% svn log -v -r 7000 http://svn.rubyonrails.org/rails trunk ------------------------------------------------------------------------ r7000 | bitsweat | 2007-06-11 10:06:07 +0200 (Mon, 11 Jun 2007) | 1 line Changed paths: M /trunk/activerecord/lib/active_record/base.rb
Remove deprecated quote methods, replaced by quote_value so quote can be used as an attribute name. ------------------------------------------------------------------------
Verbose in a range of 1000 is fine
Jasons-15er% svn log -v -r 1000:2000 http://svn.rubyonrails.org/rails trunk | wc -l 7205
Let's now extend that range, the above is 1000, let's keep increasing that by 1000 until it "breaks" again
Jasons-15er% svn log -v -r 1:2000 http://svn.rubyonrails.org/rails trunk | wc -l svn: REPORT request failed on '/rails/!svn/bc/2000' svn: REPORT of '/rails/!svn/bc/2000': Response exceeded maximum number of header fields. (http://svn.rubyonrails.org) 0
And yes the breakpoint is the size of the header fields coming back.
Let's turn the extraneous headers off-and-on.
It turns out to be the standard Header add MS-Author-Via "DAV" that one adds in (and used to be a DAV default) for some window's clients.
header being sent
Jasons-15er% svn log -v http://svn.rubyonrails.org/rails/trunk svn: REPORT request failed on '/rails/!svn/bc/7514/trunk' svn: REPORT of '/rails/!svn/bc/7514/trunk': Response exceeded maximum number of header fields. (http://svn.rubyonrails.org)
header off
Jasons-15er% svn log -v http://svn-commit.rubyonrails.org/rails/trunk > wc -l 46380
header being sent
Jasons-15er% svn log -v http://svn.rubyonrails.org/rails/trunk svn: REPORT request failed on '/rails/!svn/bc/7514/trunk' svn: REPORT of '/rails/!svn/bc/7514/trunk': Response exceeded maximum number of header fields. (http://svn.rubyonrails.org)
header off
Jasons-15er% svn log -v http://svn.rubyonrails.org/rails/trunk | wc -l
46380
The header is off (and for reference one can just turn off mod_headers entirely) and should be fine now.
Regards, Jason
Thank you very much Jason...!
It seems all to work fine now... 
As a side note, I'm not sure how mod_headers and DAV internally works, but to me sending repeated, exact duplicate headers for numerous number of times (up to the number of retrieved revisions?) seems a bit too redundant.... especially considering bandwidth isn't that cheap... But I'm not in the position to criticize this... I don't even know how it works.
Again, thanks Jason, and thank you Joyent
I am sorry I am not sure how turn the header off. Is this in Apache? thanks
I for one am glad rails has avoided threading, but find the multi-
mongrel approach a little lacking. As a twisted python user, I find
the asynch approach far superior to threading for network tasks.
Twisted apps I have built handle tens of thousands of requests per
second.
An example in ruby would be resizing an image stored on s3 - the
twisted way is to launch the http get, then add a callback that gets
invoked when the download is complete, thus allowing the server to
serve many other requests in the meantime.
Anyone else thinking in this direction?
M
I remember Twisted Python being really cool back in my Python days (even met a committer at a lan party one, heh).
I think the thing to do would be to create a similar ruby web server that supports the Rack interface. That's the approach taken with other projects like Thin and Ebb: both of which look really cool.
Keep up to date with Rails on Twitter and This Week in Rails
Policies: Conduct, License, Maintenance, Security, Trademarks