ERROR on Rails Subversion repository

While diagnosing this problem with Jelmer Vernooij (Bazaar), it seems that there is a certain "misconfiguration" on the part of Rails Subversion server hosted on Joyent.

If this is a Joyent-side issue, would someone with authority of the Rails SVN server please contact Joyent about this problem so we can resolve it?

Thank you.

Rails Subversion repository])

Hi Hendy,

There aren't any configurations to do with SVN other than turning it
on. What's happening is no different than standard HTTP errors when
you're trying to push or pull too much stuff. No different then trying
to put a string that's over 1024 bytes into IE, or trying to pass a
string that's larger to 4096 bytes to Apache (so something like
passing 200 facebook "friends" IDs in a string.)

Yes, I see the same thing with pulling up a PATH on the URL of the svn
repository

Jasons-15er% svn log -v http://svn.rubyonrails.org/rails trunk
svn: REPORT request failed on '/rails/!svn/bc/7514'
svn: REPORT of '/rails/!svn/bc/7514': Response exceeded maximum number
of header fields. (http://svn.rubyonrails.org)

The error isn't from subversion it's from the Neon library that
subversion uses for http and webdav access to stuff.

In ne_request.c, the max header is 8192 bytes

#define MAX_HEADER_LEN (8192)

/* Read response headers. Returns NE_* code, sets session error and
* closes connection on error. */
static int read_response_headers(ne_request *req)
{
    char hdr[MAX_HEADER_LEN];
    int ret, count = 0;

    while ((ret = read_message_header(req, hdr, sizeof hdr)) ==
NE_RETRY
     && ++count < MAX_HEADER_FIELDS) {
  char *pnt;
  unsigned int hash = 0;

  /* Strip any trailing whitespace */
  pnt = hdr + strlen(hdr) - 1;
  while (pnt > hdr && (*pnt == ' ' || *pnt == '\t'))
      *pnt-- = '\0';

  /* Convert the header name to lower case and hash it. */
  for (pnt = hdr; (*pnt != '\0' && *pnt != ':' &&
       *pnt != ' ' && *pnt != '\t'); pnt++) {
      *pnt = ne_tolower(*pnt);
      hash = HH_ITERATE(hash,*pnt);
  }

  /* Skip over any whitespace before the colon. */
  while (*pnt == ' ' || *pnt == '\t')
      *pnt++ = '\0';

  /* ignore header lines which lack a ':'. */
  if (*pnt != ':')
      continue;

  /* NUL-terminate at the colon (when no whitespace before) */
  *pnt++ = '\0';

  /* Skip any whitespace after the colon... */
  while (*pnt == ' ' || *pnt == '\t')
      pnt++;

  /* pnt now points to the header value. */
  NE_DEBUG(NE_DBG_HTTP, "Header Name: [%s], Value: [%s]\n", hdr, pnt);
        add_response_header(req, hash, hdr, pnt);
    }

    if (count == MAX_HEADER_FIELDS)
  ret = aborted(
      req, _("Response exceeded maximum number of header fields"), 0);

    return ret;
}

Jasons-15er% svn log -v http://svn.rubyonrails.org/rails trunk
svn: REPORT request failed on '/rails/!svn/bc/7514'
svn: REPORT of '/rails/!svn/bc/7514': Response exceeded maximum number
of header fields. (http://svn.rubyonrails.org)
Jasons-15er% svn log -v http://svn-commit.rubyonrails.org/rails trunk

wc -l

   46380

So let's take a look and see

Nonverbose is fine

Jasons-15er% svn log http://svn.rubyonrails.org/rails trunk | wc -l
   23132

Oddly enough, verbose on the entire repos URL is also fine (leave off
trunk) ... try and figure out that one

Jasons-15er% svn log -v http://svn.rubyonrails.org/rails | wc -l
   66627

Verbose on a specific version is fine

Jasons-15er% svn log -v -r 7000 http://svn.rubyonrails.org/rails
trunk

jason wrote:

Hi Hendy,

There aren't any configurations to do with SVN other than turning it
on. What's happening is no different than standard HTTP errors when
you're trying to push or pull too much stuff. No different then trying
to put a string that's over 1024 bytes into IE, or trying to pass a
string that's larger to 4096 bytes to Apache (so something like
passing 200 facebook "friends" IDs in a string.)

Yes, I see the same thing with pulling up a PATH on the URL of the svn
repository

Jasons-15er% svn log -v http://svn.rubyonrails.org/rails trunk
svn: REPORT request failed on '/rails/!svn/bc/7514'
svn: REPORT of '/rails/!svn/bc/7514': Response exceeded maximum number
of header fields. (http://svn.rubyonrails.org)

The error isn't from subversion it's from the Neon library that
subversion uses for http and webdav access to stuff.

In ne_request.c, the max header is 8192 bytes

#define MAX_HEADER_LEN (8192)

/* Read response headers. Returns NE_* code, sets session error and
* closes connection on error. */
static int read_response_headers(ne_request *req)
{
    char hdr[MAX_HEADER_LEN];
    int ret, count = 0;

    while ((ret = read_message_header(req, hdr, sizeof hdr)) ==
NE_RETRY
     && ++count < MAX_HEADER_FIELDS) {
  char *pnt;
  unsigned int hash = 0;

  /* Strip any trailing whitespace */
  pnt = hdr + strlen(hdr) - 1;
  while (pnt > hdr && (*pnt == ' ' || *pnt == '\t'))
      *pnt-- = '\0';

  /* Convert the header name to lower case and hash it. */
  for (pnt = hdr; (*pnt != '\0' && *pnt != ':' &&
       *pnt != ' ' && *pnt != '\t'); pnt++) {
      *pnt = ne_tolower(*pnt);
      hash = HH_ITERATE(hash,*pnt);
  }

  /* Skip over any whitespace before the colon. */
  while (*pnt == ' ' || *pnt == '\t')
      *pnt++ = '\0';

  /* ignore header lines which lack a ':'. */
  if (*pnt != ':')
      continue;

  /* NUL-terminate at the colon (when no whitespace before) */
  *pnt++ = '\0';

  /* Skip any whitespace after the colon... */
  while (*pnt == ' ' || *pnt == '\t')
      pnt++;

  /* pnt now points to the header value. */
  NE_DEBUG(NE_DBG_HTTP, "Header Name: [%s], Value: [%s]\n", hdr, pnt);
        add_response_header(req, hash, hdr, pnt);
    }

    if (count == MAX_HEADER_FIELDS)
  ret = aborted(
      req, _("Response exceeded maximum number of header fields"), 0);

    return ret;
}

Jasons-15er% svn log -v http://svn.rubyonrails.org/rails trunk
svn: REPORT request failed on '/rails/!svn/bc/7514'
svn: REPORT of '/rails/!svn/bc/7514': Response exceeded maximum number
of header fields. (http://svn.rubyonrails.org)
Jasons-15er% svn log -v http://svn-commit.rubyonrails.org/rails trunk
> wc -l
   46380

So let's take a look and see

Nonverbose is fine

Jasons-15er% svn log http://svn.rubyonrails.org/rails trunk | wc -l
   23132

Oddly enough, verbose on the entire repos URL is also fine (leave off
trunk) ... try and figure out that one

Jasons-15er% svn log -v http://svn.rubyonrails.org/rails | wc -l
   66627

Verbose on a specific version is fine

Jasons-15er% svn log -v -r 7000 http://svn.rubyonrails.org/rails
trunk
------------------------------------------------------------------------
r7000 | bitsweat | 2007-06-11 10:06:07 +0200 (Mon, 11 Jun 2007) | 1
line
Changed paths:
   M /trunk/activerecord/lib/active_record/base.rb

Remove deprecated quote methods, replaced by quote_value so quote can
be used as an attribute name.
------------------------------------------------------------------------

Verbose in a range of 1000 is fine

Jasons-15er% svn log -v -r 1000:2000 http://svn.rubyonrails.org/rails
trunk | wc -l
    7205

Let's now extend that range, the above is 1000, let's keep increasing
that by 1000 until it "breaks" again

Jasons-15er% svn log -v -r 1:2000 http://svn.rubyonrails.org/rails
trunk | wc -l
svn: REPORT request failed on '/rails/!svn/bc/2000'
svn: REPORT of '/rails/!svn/bc/2000': Response exceeded maximum number
of header fields. (http://svn.rubyonrails.org)
       0

And yes the breakpoint is the size of the header fields coming back.

Let's turn the extraneous headers off-and-on.

It turns out to be the standard Header add MS-Author-Via "DAV" that
one adds in (and used to be a DAV default) for some window's clients.

header being sent

Jasons-15er% svn log -v http://svn.rubyonrails.org/rails/trunk
svn: REPORT request failed on '/rails/!svn/bc/7514/trunk'
svn: REPORT of '/rails/!svn/bc/7514/trunk': Response exceeded maximum
number of header fields. (http://svn.rubyonrails.org)

header off

Jasons-15er% svn log -v http://svn-commit.rubyonrails.org/rails/trunk
> wc -l
   46380

header being sent

Jasons-15er% svn log -v http://svn.rubyonrails.org/rails/trunk
svn: REPORT request failed on '/rails/!svn/bc/7514/trunk'
svn: REPORT of '/rails/!svn/bc/7514/trunk': Response exceeded maximum
number of header fields. (http://svn.rubyonrails.org)

header off

Jasons-15er% svn log -v http://svn.rubyonrails.org/rails/trunk | wc -l

   46380

The header is off (and for reference one can just turn off mod_headers
entirely) and should be fine now.

Regards, Jason

Thank you very much Jason...!

It seems all to work fine now... :slight_smile:

As a side note, I'm not sure how mod_headers and DAV internally works,
but to me sending repeated, exact duplicate headers for numerous number
of times (up to the number of retrieved revisions?) seems a bit too
redundant.... especially considering bandwidth isn't that cheap... But
I'm not in the position to criticize this... I don't even know how it works.

Again, thanks Jason, and thank you Joyent :slight_smile:

I am sorry I am not sure how turn the header off. Is this in Apache?
thanks

I for one am glad rails has avoided threading, but find the multi-
mongrel approach a little lacking. As a twisted python user, I find
the asynch approach far superior to threading for network tasks.
Twisted apps I have built handle tens of thousands of requests per
second.

An example in ruby would be resizing an image stored on s3 - the
twisted way is to launch the http get, then add a callback that gets
invoked when the download is complete, thus allowing the server to
serve many other requests in the meantime.

Anyone else thinking in this direction?

M

I remember Twisted Python being really cool back in my Python days
(even met a committer at a lan party one, heh).

I think the thing to do would be to create a similar ruby web server
that supports the Rack interface. That's the approach taken with other
projects like Thin and Ebb: both of which look really cool.