Page rendering dies halfway but rails thinks it has finished

Hi all. I've got an odd problem which occurs on our server but doesn't
happen locally. (On the server we have nginx with a mongrel cluster
underneath)

I have a request which takes a large amount of text from a text field,
shoves it into a background process (done with spawn) to be processed,
and then renders out a progress page. The progress page is then updated
with ajax which checks how the spawned task is getting on.

The problem is, i think, nothing to do with spawn though. What's
happening is that the progress page is not fully rendered - looking at
the page source, it ends abruptly, ALWAYS AT THE SAME POINT for any
given data import. This occurs before any of the ajax stuff kicks off:
the basic page is truncated.

So, the page source will end like this - cut off halfway through a tag:

        <span id="qid_1944" class="qid incomplete">1944</span>
        <span id="qid_1945" class="qid incomplete">1945</span>
        <span id="qid_1946" class="qid incomplete">1946</span>
        <span id="qid_1947" class="qid incomplet

Looking in the log file, as far as rails is concerned it has completed
the page render quite happily, and doesn't report any problems. It's as
if it has sent it back to the browser but the browser only gets some of
it, not all. Or maybe the browser gets all of it but it's broken in
such a way as to make the browser only display some of it.

I get the same thing happening in firefox and chrome: it dies at the
same point in each. However, it's not always the same tag that it
breaks in for different imported data. For example, let's say i import
two chunks of text, both of which have the '1947' element that it dies
in above. The first one always dies in 1947, the other one renders 1947
ok but dies in 1950. So, import A always dies rendering 1947, import B
always dies in 1950.

If it was a timeout thing i wouldn't expect it to be so consistent - it
always dies at the exact same character every time. Similarly, i
thought it might be due to some data limit, but if I half the amount of
text that is being imported, it still dies: at a different point to the
import above, but once again always the same point every time for that
given data import.

This is driving me nuts. Can anyone offer any advice/explanation? I'm
grateful for any...

max

I'd start by using something like tcpdump, wireshark etc to sniff the
network traffic in order to work out which of those possibilities is
true (also look on the server - is mongrel actually sending out all
the bytes it says it is? is nginx forwarding them on?)
Is the progress page particularly big?

Fred

You might be using some C extension that is crushing ?
Anything in nginx or mongrel log files ?