Should chunked encoding (by Rails) be optional for HTTP streaming?

Hi Tim!

Hello,

I have been using the HTTP streaming feature for my company's website for a
few months now. It has significantly decreased the page load times for our
asset-heavy pages.

I have encountered a few problems with HTTP streaming which can be traced
to Rails always doing the chunked encoding itself:

- Incompatible middleware such as newrelic (granted it's easy to solve by
putting the newrelic header and footer in manually but there are probably
other incompatible middleware).

I'm not familiar with this, but it seems like something newrelic should
provide?

- Problems with running assertions against a response body in integration
tests. The response has chunk markers in it making it difficult to parse
the body.

Are these *rails* integration tests? If so, we should consider this a
bug. I would expect chunks to be assembled in to normal bodies in the
integration tests.

- Incompatible servers that chunk encode again resulting in a double
chunked encoding. I had to monkey patch Rails to not chunk encode in order
to get streaming working properly with JRuby / Tomcat. See
https://github.com/jruby/jruby-rack/issues/117

Not fun. :frowning:

I understand the chunked encoding is required in the end in order to
stream. But I wonder if instead of having Rails doing it at the core, it
can instead be moved to a Rack middleware that is placed at the end of the
middleware stack when needed and can be left out when the server is capable
of doing the chunked encoding. This could be controlled by a configuration
option.

As an intermediate step towards moving it to a Rack middleware, chunked
encoding could first be made optional. This would at least solve the JRuby
/ Tomcat issue.

It seems entirely possible to do this. Rack has a chunking middleware
that can be inserted in the stack. I think the main problem is that by
the time the framework learns that you want a streaming response (the
point when you call render with stream), the middleware stack is already
assembled. In order to make the streaming Just Work, you'd have to add
a chunked body right at that point.

We could eliminate the chunked body [from here][1], but it means two
things:

1. You'd have to manually add the chunking middleware for *that*
   controller.

2. It would break backwards compatibility

We could add the chunking middleware by default, but I'm not sure
everyone would want that.

I have another idea to fix this, but it's a bit more effort. AC::Live
doesn't chunk responses. I think it's possible to get the ERB to write
to a buffer that AC::Live uses. I've been looking in to this, but it
hasn't been my highest priority lately. :frowning:

[1]: https://github.com/rails/rails/blob/master/actionpack/lib/action_controller/metal/streaming.rb#L219

I have encountered a few problems with HTTP streaming which can be traced
to Rails always doing the chunked encoding itself:

  • Incompatible middleware such as newrelic (granted it’s easy to solve by
    putting the newrelic header and footer in manually but there are probably
    other incompatible middleware).

I’m not familiar with this, but it seems like something newrelic should

provide?

Maybe, but it seems a little redundant for everything along the middleware stack to have to dechunk if it wants to read the body and rechunk if it wants to modify the body.

  • Problems with running assertions against a response body in integration
    tests. The response has chunk markers in it making it difficult to parse
    the body.

Are these rails integration tests? If so, we should consider this a

bug. I would expect chunks to be assembled in to normal bodies in the

integration tests.

It’s happening for me in rails request specs, which I assume means it is happening in rails integration tests. The problem went away when I monkey patched Rails to not chunk.

I understand the chunked encoding is required in the end in order to
stream. But I wonder if instead of having Rails doing it at the core, it
can instead be moved to a Rack middleware that is placed at the end of the
middleware stack when needed and can be left out when the server is capable
of doing the chunked encoding. This could be controlled by a configuration
option.

As an intermediate step towards moving it to a Rack middleware, chunked
encoding could first be made optional. This would at least solve the JRuby
/ Tomcat issue.

It seems entirely possible to do this. Rack has a chunking middleware

that can be inserted in the stack. I think the main problem is that by

the time the framework learns that you want a streaming response (the

point when you call render with stream), the middleware stack is already

assembled. In order to make the streaming Just Work, you’d have to add

a chunked body right at that point.

How about a chunking middleware that only chunk encodes if the Content-Length header is not set? Otherwise it allows the response to pass through as is.

We could eliminate the chunked body [from here][1], but it means two

things:

  1. You’d have to manually add the chunking middleware for that

    controller.

  2. It would break backwards compatibility

The only thing I can think of breaking is middleware that expects a chunked encoding when Content-Length is not set. I would hope they would detect chunked encoding by the Transfer-Encoding header, but who knows. Is there anything else you think can break?

We could add the chunking middleware by default, but I’m not sure

everyone would want that.

If chunking middleware proves distasteful for too many, maybe just having a configuration option to turn off the chunking (for servers such as Tomcat) is the way to go.

I have another idea to fix this, but it’s a bit more effort. AC::Live

doesn’t chunk responses. I think it’s possible to get the ERB to write

to a buffer that AC::Live uses. I’ve been looking in to this, but it

hasn’t been my highest priority lately. :frowning:

Interesting. Does AC::Live just rely on the webserver to do the chunking?

Tim

Is there a problem with chunking and Tomcat? jruby-rack doesn't support
it directly, but it doesn't prohibit it either. It appears to work if
you set a response_body whose #each yields properly formatted chunks and
a terminating empty chunk. I've just started to use this to pass through
(already chunked) responses from another services.

Michael

I and others have seen Tomcat double chunk streamed responses coming from Rails. It may be the case, however, that this only happens for responses over a certain size. I’ve filed an issue with more details at https://github.com/jruby/jruby-rack/issues/117 .

How large are your responses? I’ve only tried so far to get one of our streamed pages to work in a jruby setup. It’s response is 68,019 bytes.

Tim

>
> Is there a problem with chunking and Tomcat? jruby-rack doesn't
> support it directly, but it doesn't prohibit it either. It appears
> to work if you set a response_body whose #each yields properly
> formatted chunks and a terminating empty chunk. I've just started
> to use this to pass through (already chunked) responses from
> another services.

I and others have seen Tomcat double chunk streamed responses coming
from Rails. It may be the case, however, that this only happens for
responses over a certain size. I've filed an issue with more
details at https://github.com/jruby/jruby-rack/issues/117 .

I think our cases are different. I'm not using the Rails streaming
support at all. Instead, I'm setting the necessary headers and the
response_body myself. In particular, I'm setting response_body to an
object implementing #each in such a way that it yields properly
formatted chunks. jruby-rack writes these chunks individually to the
servlet's output stream. I haven't yet tested this exhaustively, but as
far as I can tell, Tomcat (or rather Winstone in my tests) simply passes
the chunks through to its browser client in turn. That is exactly the
behavior I want, as my Rails app in this case only proxies attachments
stored by another, internal webservice.

How large are your responses? I've only tried so far to get one of
our streamed pages to work in a jruby setup. It's response is
68,019 bytes.

I don't think our cases are comparable. I've tried my mechanism on cases
where the document body is at most 1MB; but I haven't seen any
indication that it would work differently for much larger or smaller
responses.

Michael

I think our cases are different. I’m not using the Rails streaming

support at all. Instead, I’m setting the necessary headers and the

response_body myself. In particular, I’m setting response_body to an

object implementing #each in such a way that it yields properly

formatted chunks. jruby-rack writes these chunks individually to the

servlet’s output stream. I haven’t yet tested this exhaustively, but as

far as I can tell, Tomcat (or rather Winstone in my tests) simply passes

the chunks through to its browser client in turn. That is exactly the

behavior I want, as my Rails app in this case only proxies attachments

stored by another, internal webservice.

By chunks do you mean HTTP/1.1 chunks [1]? In other words, do you set the Transfer-Encoding to “chunked” and are your chunks prefixed by the size of the chunk in hex and a CRLF? If not, then that is not HTTP chunked encoding. It’s by removing the chunked encoding (the header, the size in hex and CRLF) that I was able to get things working properly with Tomcat.

As for Winstone, I have tried that server and it works with the chunked encoding.

I don’t think our cases are comparable. I’ve tried my mechanism on cases

where the document body is at most 1MB; but I haven’t seen any

indication that it would work differently for much larger or smaller

responses.

With that size, I would expect you to see double chunked encoding. Double chunked encoding results in the hex showing through, that is if you’re not using a browser such as Chrome which refuses to display a page with two Transfer-Encoding headers.

Tim

[1] http://greenbytes.de/tech/webdav/rfc2616.html#chunked.transfer.encoding

> I think our cases are different. I'm not using the Rails streaming
> support at all. Instead, I'm setting the necessary headers and the
> response_body myself. In particular, I'm setting response_body to
> an object implementing #each in such a way that it yields properly
> formatted chunks. jruby-rack writes these chunks individually to
> the servlet's output stream. I haven't yet tested this
> exhaustively, but as far as I can tell, Tomcat (or rather Winstone
> in my tests) simply passes the chunks through to its browser
> client in turn. That is exactly the behavior I want, as my Rails
> app in this case only proxies attachments stored by another,
> internal webservice.

By chunks do you mean HTTP/1.1 chunks [1]? In other words, do you
set the Transfer-Encoding to "chunked" and are your chunks prefixed
by the size of the chunk in hex and a CRLF?

Yes. I do this myself in the #each method of the object to which I set
response_body.

If not, then that is
not HTTP chunked encoding. It's by removing the chunked encoding
(the header, the size in hex and CRLF) that I was able to get things
working properly with Tomcat.

As for Winstone, I have tried that server and it works with the
chunked encoding.

Ouch. I mistakenly thought that Winstone was an embedded variant of
Tomcat. I was probably confusing it with Trinidad. Anyway, for
Tomcat/Trinidad, now do not set the Transfer-Encoding header and I just
yield the data then Tomcat does the right thing. It does not use the
exact blocks of data for chunks that I yield, but it generates properly
formatted chunks and a trailing empty chunk.

Michael

If I am understanding you correctly, you are not sending chunked data to Tomcat and Tomcat does the chunking. The problem I’m having is that Tomcat chunks again the already chunked data coming from Rails streaming. So my solution is similar to yours (if I’m understanding you correctly): I monkey patched Rails to not send chunked data to Tomcat and Tomcat takes care of the chunking.

Tim