Tracked this down to a problem with x_sendfile - i needed to set it to false in order to get it working. Here's my current call:
render_pdf :template => 'school_subscribers/show_pdf', :send_file => { :filename => filename, :x_sendfile => false, :disposition => "inline", :type => "application/pdf"}
I don't know why x_sendfile is going wrong - it misbehaves the same way in my local single mongrel setup, my local apache&mongrel setup and our staging server which is apache with multiple mongrels. I don't know much about x_sendfile but even my much more knowledgeable colleague is stumped.