Should chunked encoding (by Rails) be optional for HTTP streaming?

rubyonrails-core
1

Hi Tim!

Hello,

I have been using the HTTP streaming feature for my company's website for a few months now. It has significantly decreased the page load times for our asset-heavy pages.

I have encountered a few problems with HTTP streaming which can be traced to Rails always doing the chunked encoding itself:

- Incompatible middleware such as newrelic (granted it's easy to solve by putting the newrelic header and footer in manually but there are probably other incompatible middleware).

I'm not familiar with this, but it seems like something newrelic should provide?

- Problems with running assertions against a response body in integration tests. The response has chunk markers in it making it difficult to parse the body.

Are these *rails* integration tests? If so, we should consider this a bug. I would expect chunks to be assembled in to normal bodies in the integration tests.

- Incompatible servers that chunk encode again resulting in a double chunked encoding. I had to monkey patch Rails to not chunk encode in order to get streaming working properly with JRuby / Tomcat. See rails streamed pages are doubly chunked encoded after going through tomcat · Issue #117 · jruby/jruby-rack · GitHub

Not fun. :frowning:

I understand the chunked encoding is required in the end in order to stream. But I wonder if instead of having Rails doing it at the core, it can instead be moved to a Rack middleware that is placed at the end of the middleware stack when needed and can be left out when the server is capable of doing the chunked encoding. This could be controlled by a configuration option.

As an intermediate step towards moving it to a Rack middleware, chunked encoding could first be made optional. This would at least solve the JRuby / Tomcat issue.

It seems entirely possible to do this. Rack has a chunking middleware that can be inserted in the stack. I think the main problem is that by the time the framework learns that you want a streaming response (the point when you call render with stream), the middleware stack is already assembled. In order to make the streaming Just Work, you'd have to add a chunked body right at that point.

We could eliminate the chunked body [from here][1], but it means two things:

1. You'd have to manually add the chunking middleware for *that*    controller.

2. It would break backwards compatibility

We could add the chunking middleware by default, but I'm not sure everyone would want that.

I have another idea to fix this, but it's a bit more effort. AC::Live doesn't chunk responses. I think it's possible to get the ERB to write to a buffer that AC::Live uses. I've been looking in to this, but it hasn't been my highest priority lately. :frowning:

[1]: <%= yield :title %>

2

I have encountered a few problems with HTTP streaming which can be traced to Rails always doing the chunked encoding itself:

  • Incompatible middleware such as newrelic (granted it’s easy to solve by putting the newrelic header and footer in manually but there are probably other incompatible middleware).

I’m not familiar with this, but it seems like something newrelic should

provide?

Maybe, but it seems a little redundant for everything along the middleware stack to have to dechunk if it wants to read the body and rechunk if it wants to modify the body.

  • Problems with running assertions against a response body in integration tests. The response has chunk markers in it making it difficult to parse the body.

Are these rails integration tests? If so, we should consider this a

bug. I would expect chunks to be assembled in to normal bodies in the

integration tests.

It’s happening for me in rails request specs, which I assume means it is happening in rails integration tests. The problem went away when I monkey patched Rails to not chunk.

I understand the chunked encoding is required in the end in order to stream. But I wonder if instead of having Rails doing it at the core, it can instead be moved to a Rack middleware that is placed at the end of the middleware stack when needed and can be left out when the server is capable of doing the chunked encoding. This could be controlled by a configuration option.

As an intermediate step towards moving it to a Rack middleware, chunked encoding could first be made optional. This would at least solve the JRuby / Tomcat issue.

It seems entirely possible to do this. Rack has a chunking middleware

that can be inserted in the stack. I think the main problem is that by

the time the framework learns that you want a streaming response (the

point when you call render with stream), the middleware stack is already

assembled. In order to make the streaming Just Work, you’d have to add

a chunked body right at that point.

How about a chunking middleware that only chunk encodes if the Content-Length header is not set? Otherwise it allows the response to pass through as is.

We could eliminate the chunked body [from here][1], but it means two

things:

  1. You’d have to manually add the chunking middleware for that

    controller.

  2. It would break backwards compatibility

The only thing I can think of breaking is middleware that expects a chunked encoding when Content-Length is not set. I would hope they would detect chunked encoding by the Transfer-Encoding header, but who knows. Is there anything else you think can break?

We could add the chunking middleware by default, but I’m not sure

everyone would want that.

If chunking middleware proves distasteful for too many, maybe just having a configuration option to turn off the chunking (for servers such as Tomcat) is the way to go.

I have another idea to fix this, but it’s a bit more effort. AC::Live

doesn’t chunk responses. I think it’s possible to get the ERB to write

to a buffer that AC::Live uses. I’ve been looking in to this, but it

hasn’t been my highest priority lately. :frowning:

Interesting. Does AC::Live just rely on the webserver to do the chunking?

Tim

3

Is there a problem with chunking and Tomcat? jruby-rack doesn't support it directly, but it doesn't prohibit it either. It appears to work if you set a response_body whose #each yields properly formatted chunks and a terminating empty chunk. I've just started to use this to pass through (already chunked) responses from another services.

Michael

4

I and others have seen Tomcat double chunk streamed responses coming from Rails. It may be the case, however, that this only happens for responses over a certain size. I’ve filed an issue with more details at https://github.com/jruby/jruby-rack/issues/117 .

How large are your responses? I’ve only tried so far to get one of our streamed pages to work in a jruby setup. It’s response is 68,019 bytes.

Tim

5

> > Is there a problem with chunking and Tomcat? jruby-rack doesn't > support it directly, but it doesn't prohibit it either. It appears > to work if you set a response_body whose #each yields properly > formatted chunks and a terminating empty chunk. I've just started > to use this to pass through (already chunked) responses from > another services.

I and others have seen Tomcat double chunk streamed responses coming from Rails. It may be the case, however, that this only happens for responses over a certain size. I've filed an issue with more details at rails streamed pages are doubly chunked encoded after going through tomcat · Issue #117 · jruby/jruby-rack · GitHub .

I think our cases are different. I'm not using the Rails streaming support at all. Instead, I'm setting the necessary headers and the response_body myself. In particular, I'm setting response_body to an object implementing #each in such a way that it yields properly formatted chunks. jruby-rack writes these chunks individually to the servlet's output stream. I haven't yet tested this exhaustively, but as far as I can tell, Tomcat (or rather Winstone in my tests) simply passes the chunks through to its browser client in turn. That is exactly the behavior I want, as my Rails app in this case only proxies attachments stored by another, internal webservice.

How large are your responses? I've only tried so far to get one of our streamed pages to work in a jruby setup. It's response is 68,019 bytes.

I don't think our cases are comparable. I've tried my mechanism on cases where the document body is at most 1MB; but I haven't seen any indication that it would work differently for much larger or smaller responses.

Michael

6

I think our cases are different. I’m not using the Rails streaming

support at all. Instead, I’m setting the necessary headers and the

response_body myself. In particular, I’m setting response_body to an

object implementing #each in such a way that it yields properly

formatted chunks. jruby-rack writes these chunks individually to the

servlet’s output stream. I haven’t yet tested this exhaustively, but as

far as I can tell, Tomcat (or rather Winstone in my tests) simply passes

the chunks through to its browser client in turn. That is exactly the

behavior I want, as my Rails app in this case only proxies attachments

stored by another, internal webservice.

By chunks do you mean HTTP/1.1 chunks [1]? In other words, do you set the Transfer-Encoding to “chunked” and are your chunks prefixed by the size of the chunk in hex and a CRLF? If not, then that is not HTTP chunked encoding. It’s by removing the chunked encoding (the header, the size in hex and CRLF) that I was able to get things working properly with Tomcat.

As for Winstone, I have tried that server and it works with the chunked encoding.

I don’t think our cases are comparable. I’ve tried my mechanism on cases

where the document body is at most 1MB; but I haven’t seen any

indication that it would work differently for much larger or smaller

responses.

With that size, I would expect you to see double chunked encoding. Double chunked encoding results in the hex showing through, that is if you’re not using a browser such as Chrome which refuses to display a page with two Transfer-Encoding headers.

Tim

[1] http://greenbytes.de/tech/webdav/rfc2616.html#chunked.transfer.encoding

7

> I think our cases are different. I'm not using the Rails streaming > support at all. Instead, I'm setting the necessary headers and the > response_body myself. In particular, I'm setting response_body to > an object implementing #each in such a way that it yields properly > formatted chunks. jruby-rack writes these chunks individually to > the servlet's output stream. I haven't yet tested this > exhaustively, but as far as I can tell, Tomcat (or rather Winstone > in my tests) simply passes the chunks through to its browser > client in turn. That is exactly the behavior I want, as my Rails > app in this case only proxies attachments stored by another, > internal webservice.

By chunks do you mean HTTP/1.1 chunks [1]? In other words, do you set the Transfer-Encoding to "chunked" and are your chunks prefixed by the size of the chunk in hex and a CRLF?

Yes. I do this myself in the #each method of the object to which I set response_body.

If not, then that is not HTTP chunked encoding. It's by removing the chunked encoding (the header, the size in hex and CRLF) that I was able to get things working properly with Tomcat.

As for Winstone, I have tried that server and it works with the chunked encoding.

Ouch. I mistakenly thought that Winstone was an embedded variant of Tomcat. I was probably confusing it with Trinidad. Anyway, for Tomcat/Trinidad, now do not set the Transfer-Encoding header and I just yield the data then Tomcat does the right thing. It does not use the exact blocks of data for chunks that I yield, but it generates properly formatted chunks and a trailing empty chunk.

Michael

8

If I am understanding you correctly, you are not sending chunked data to Tomcat and Tomcat does the chunking. The problem I’m having is that Tomcat chunks again the already chunked data coming from Rails streaming. So my solution is similar to yours (if I’m understanding you correctly): I monkey patched Rails to not send chunked data to Tomcat and Tomcat takes care of the chunking.

Tim