How do I add intensive processes to my Rails Application?

I’ve started to use wicked_pdf and it works fine. However, when I generate a PDF files, usually the CPU will get to ~80+% during the generation of the PDF by the wicked_pdf gem. What’s the proper way of handling this in my application? I’ve asked around on IRC chat, stack overflow, etc, but I haven’t been able to acquire a good answer. People have told me to use Sidekiq, which I have, but this won’t resolve my CPU issue unless I’ve put the job processes onto another machine it would seem. Is there another way of handling this using my one server?

If it’s helpful here is the stack overflow question that I posted about 12 days ago - it has perhaps a little more detail:

https://stackoverflow.com/questions/44465332/high-cpu-usage-when-generating-pdfs-in-rails-with-wicked-pdf-gem

Please let me know if I can provide more details on any aspect of this issue. Thanks!

Ouch! The big question is: are you actually doing anything else with that server? If Rails has the machine all to itself, and you don't have enough traffic that someone else is probably waiting for a reply, you may be OK as-is.

Otherwise, a number of other approaches come to mind. As you mentioned, you could put it on another machine. You could also see if you could "nice down" that process, i.e., use the "nice" utility (or some such programming) to set the priority of the PDF-making process so that others can have a better share.

-Dave

Thanks Dave, I’ll look into the “Nice” utility you mentioned – I’m currently unfamiliar with it. Also traffic on the application isn’t heavy as its only used on an internal network for roughly 60 users at the moment. However, I would prefer to find the “correct” way of doing it. I’m thinking I may resort to using another machine, but at the moment I don’t have those resources. Rails is the only thing on the server right now… so that helps as well. With some of the PDFs that time several minutes to run, we have seen a slow-down for other users accessing pages. Since the PDF generation isn’t urgent I don’t mind having it last longer as long as it doesn’t slow down the rest of the application for the other active users. Thanks again for the “Nice” recommendation I’ll be looking into it shortly.

This is exactly the kind of use case AWS Lambda was made for.

You would have to switch to a PDF generator/converter in one of the Lambda-supported languages, unfortunately, but for a very occasional task this would be a lot cheaper than provisioning a separate server that would be idle most of the time.

FWIW,

Why is the fact that PDF generation can use 80% of the processor a problem? There will still be 20% available for Rails, or is that not enough to give you the performance you require? Linux is pretty good at balancing processes and threads and the processor is there to be used after all. Provided you have shelved it off into a background process using something like sidekiq then it may well not be an issue. Try it and see. if it becomes an issue then you can look at more complex solutions. My philosophy is always to start off using the KISS principle.

Colin

Hey Colin, when I say it gets to ~80+% what I meant to get across is, for a 1 minute PDF Generation, it would start around 20% and quickly built to 80 - then 95 - and eventually go to 100% towards the end. I’ve noticed that many other users will have the application slow down dramatically during the generation of such a PDF. It would cause much longer page loads and if they too were going to generate a PDF document then it would make problems worse. I have been attempting to background the process on Sidekiq - and am having another issue with it at the moment preventing me from seeing how it behaves in that procedure. Though with it all using the same CPU I was guessing it wouldn’t change much from what I’m already experiencing.

OK, understood. I think it should be possible to set the nice value for sidekiq so that it only uses the processor when it is not doing other things, but I don't have experience of doing that, and google does not immediately come up with anything helpful.

Colin