Detecting ActionController::Live/streaming from a middleware?

Jason_Clark1 · July 14, 2014, 9:14pm

I’m curious if there’s a good way to detect from a Rack middleware whether a response is streaming or not.

From what I can see, my options are:

In the Controller note we’ve included ActionController::Live somewhere the middleware can access it
Look at the response object coming back to Rack to see if its class is ActionController::Live::Response
Look at the stream (if available) on the response object returned to Rack to check if it’s a ActionController::Live::Buffer

All of these ideas seem pretty brittle and liable to break across new versions of Rails and/or new streaming mechanisms that might come along. Is there something I’m missing?

Thanks!

Jason Clark

@jasonrclark

New Relic, Ruby Agent engineer

ismaSan · July 15, 2014, 11:58am

I haven’t tested this but an HTTP streaming response should not have a Content-Length header. Maybe that’s enough to detect whether it’s streaming or not? So you look at the properties of the HTTP response itself instead of the Ruby wrapper, which as you say might change in the future.

Rodrigo_Rosenfeld_R1 · July 15, 2014, 12:38pm

This would depend if Content-Length would be set by another middleware, so depending on the order of the middlewares it might work or not.

This is just as brittle in my opinion. The proper way of handling this is through some new API in the Rack library itself to let the middlewares know whether it's a streamed response or not.

Such API would also be useful in this issue:

https://github.com/rails/rails/issues/14358

By the way, I can't find it by searching on GitHub. Any ideas why? I can only access it because I had the link stored in some e-mail of my inbox...

Rodrigo_Rosenfeld_R1 · July 15, 2014, 12:44pm

nevermind, it seems there were some filters in place filtering the issues which I found out later. Sorry for the distraction.

Rodrigo_Rosenfeld_R1 · July 15, 2014, 3:20pm

Hi Jason,

Actually (2) is not possible using the public API if I remember correctly because the response is wrapped inside a Rack::BodyProxy (or something like that). That's why I used (3) when I needed to detect if streaming was enabled.

Another option would be to use the same convention as Rack::ETag middleware and do not attempt to read the response if the Cache-Control header is set to no-cache. ActionController::Live will set this header on the first response.stream.write call (although there's still a bug that will happen and affect the ETag middleware as well if the action calls response.commit! before response.stream.write).

This might be a better and less intrusive work-around until Rack support some new API to let the middlewares know whether it's a streamed response or not.

HTH, Rodrigo.

Rodrigo_Rosenfeld_R1 · July 15, 2014, 8:03pm

Just to add some context about this in case it helps to get some feedback:

NewRelic has an interesting feature for Ruby applications called auto-instrumentation, which will inject the required code in the HTML header in a Rack middleware.

For this to work, it has to read the response body and then modify it accordingly. But it's not safe (or desirable) to do so for streamed responses. Not only it would remove the benefits from streaming but it may also cause bugs in applications that would otherwise work without that middleware (if auto-instrumentation is enabled, which is usually the case). For instance, Rails will freeze the headers hash for streaming responses after the first commit/flush, since the headers have already been sent. Another middleware could be tempted to add some header (cookie, whatever) and will then get an exception when trying to do so.

This is not a theoretical issue. For my application, the NewRelic middleware caused the flush to happen before the other Rack middleware would set the cookies.

So it would be extremely interesting if Rack (or Rails at least) could provide some reliable supported way for middlewares to know whether a response is streamed or not. Should we use the same convention as the Rack::ETag middleware and check for Cache-Control: no-cache? Or is there another recommended method for doing so?

Thanks in advance, Rodrigo.

ismaSan · July 16, 2014, 12:07pm

Oh I totally agree that a proper API in Rack would be better for this. But I thought the problem is precisely that there isn’t one?

I can see that Rack already knows whether the response is “chunked” or not (which I think is what “streaming” means in Rails, as opposed to Server Sent Events, Websockets and other ways of streaming data over HTTP).

github.com

rack/rack/blob/main/lib/rack/response.rb#L27


      
          # You can use Response#write to iteratively generate your response,
          # but note that this is buffered by Rack::Response until you call
          # +finish+.  +finish+ however can take a block inside which calls to
          # +write+ are synchronous with the Rack response.
          #
          # Your application's +call+ should end returning Response#finish.
          class Response
            def self.[](status, headers, body)
              self.new(body, status, headers)
            end
          
          
  CHUNKED = 'chunked'
            STATUS_WITH_NO_ENTITY_BODY = Utils::STATUS_WITH_NO_ENTITY_BODY
          
          
  attr_accessor :length, :status, :body
            attr_reader :headers
          
          
  # Initialize the response object with the specified +body+, +status+
            # and +headers+.
            #
            # If the +body+ is +nil+, construct an empty response object with internal

So maybe the only thing that needs to be added is:

def streaming?

@chunked

end

Rodrigo_Rosenfeld_R1 · July 16, 2014, 5:06pm

Well, this is the implementation for the link you provided:

@chunked = "chunked" == @header['Transfer-Encoding']

This would be equivalent for the Rack middleware to check for this particular header. But this is highly dependent on whether the Rails action will set this header or some other middleware present in the application's stack. For my particular action this header is not present for instance. If all streamed responses are supposed to set this header, than Rails should include it for all controller's actions where ActionController::Live is included.

Jason_Clark1 · July 17, 2014, 4:27pm

Hey Rodrigo

Actually (2) is not possible using the public API if I remember correctly because the response is wrapped inside a Rack::BodyProxy (or something like that). That’s why I used (3) when I needed to detect if streaming was enabled.

Good point! I hadn’t noticed that in my experimenting, but that makes sense.

Another option would be to use the same convention as Rack::ETag middleware and do not attempt to read the response if the Cache-Control header is set to no-cache. ActionController::Live will set this header on the first response.stream.write call (although there’s still a bug that will happen and affect the ETag middleware as well if the action calls response.commit! before response.stream.write).

Interesting idea about using the Cache-Control header.

Thanks for the additional input and ideas!

~Jason