Page rendering dies halfway but rails thinks it has finished

Hi all. I've got an odd problem which occurs on our server but doesn't happen locally. (On the server we have nginx with a mongrel cluster underneath)

I have a request which takes a large amount of text from a text field, shoves it into a background process (done with spawn) to be processed, and then renders out a progress page. The progress page is then updated with ajax which checks how the spawned task is getting on.

The problem is, i think, nothing to do with spawn though. What's happening is that the progress page is not fully rendered - looking at the page source, it ends abruptly, ALWAYS AT THE SAME POINT for any given data import. This occurs before any of the ajax stuff kicks off: the basic page is truncated.

So, the page source will end like this - cut off halfway through a tag:

        <span id="qid_1944" class="qid incomplete">1944</span>         <span id="qid_1945" class="qid incomplete">1945</span>         <span id="qid_1946" class="qid incomplete">1946</span>         <span id="qid_1947" class="qid incomplet

Looking in the log file, as far as rails is concerned it has completed the page render quite happily, and doesn't report any problems. It's as if it has sent it back to the browser but the browser only gets some of it, not all. Or maybe the browser gets all of it but it's broken in such a way as to make the browser only display some of it.

I get the same thing happening in firefox and chrome: it dies at the same point in each. However, it's not always the same tag that it breaks in for different imported data. For example, let's say i import two chunks of text, both of which have the '1947' element that it dies in above. The first one always dies in 1947, the other one renders 1947 ok but dies in 1950. So, import A always dies rendering 1947, import B always dies in 1950.

If it was a timeout thing i wouldn't expect it to be so consistent - it always dies at the exact same character every time. Similarly, i thought it might be due to some data limit, but if I half the amount of text that is being imported, it still dies: at a different point to the import above, but once again always the same point every time for that given data import.

This is driving me nuts. Can anyone offer any advice/explanation? I'm grateful for any...

max

I'd start by using something like tcpdump, wireshark etc to sniff the network traffic in order to work out which of those possibilities is true (also look on the server - is mongrel actually sending out all the bytes it says it is? is nginx forwarding them on?) Is the progress page particularly big?

Fred

You might be using some C extension that is crushing ? Anything in nginx or mongrel log files ?