How to deal with user requests that take time to process?

Hi guys,

Could you recommend a common design for the following case (or point
me in the right direction):

- the user makes a request
- the server processes is it, but the process is long therefore it is
split into "chunks".
- the page is rendered with the first chunk
- when another chunk is ready, it is pushed to the client (say at the
bottom of the page or something).

I guess a solution could be:
- do the process in the background, store the chunks in a DB (marked
using session)
- get periodic AJAX calls from the client to check whether there is
"something new available" in the DB
- if there is, push to the client.

But it seems a bit heavy (and could potentially be thread unsafe?) to
me. Besides, it would be better to push the chunks as soon as they are
ready rather than waiting for periodic calls.

Is there a preferred way to do that?

Thanks a lot,

Juggernaut (Flash based pushing) or Comet (persistent HTTP connection pushing). Doubt there’s a lot of prechewed code for either of them out there though.

Polling isn’t really that bad in this case imo. When there is no chunk, your app will just return a status, when there is, it returns data. That’s less actual data than Campfire has to deal with and they are using polling.

Best regards

Peter De Berdt