Making rails thread safe, priority or not?

rubyonrails-talk
1

I know there has been a lot of talk about rails and thread safety. Maybe I've missed it, but I havent' seen any of the core team talk about whether they plan on fixing this in the near future or not. It seems to me that making rails thread safe would do more to enhance rails then any other single thing that could be done. I'm not under any illusions, I'm sure it would be a lot of work. But I have to ask myself the question. If it's something that will have to be done at some point, and I think it will, doesn't it make sense to tackle it sooner rather than later?

So what do the users think? Would you be willing to see work on other things stopped to get this fixed?

Also, this isn't a jab at the developers, I just haven't seen anyone ask the question like this before. I've seen lots of discussion about how things are and particular areas in rails that are problematic, but no statements as to what might be done about it looking towards the future or what the average rails user thinks about it.

And I will personally volunteer some of my time to help out if the decision is made to do something about this.

Chris

2

Yeah I would love to see rails become thread safe as well. I have no idea how big a task that would be though. I think it might be monumental :wink: I would donate some time to help with that as well. But I fear it would require a huge rewrite of a lot of stuf and I am not sure if it is worth the huge amount of work it might be. Might be better to just build a different thread safe ActionPack clone.

-Ezra

3

Yes I've seen that, good stuff:) The challenge is that there is probably so much to be done, that short of the core team deciding as a group that they will make it a priority, it simply won't get done. Even if a heroic individual decided to step up and do all the work, it would have to be in conjunction with new rules that didn't let in new code that wasn't thread safe.

Even without being thread safe the world isn't going to end, but if it were thread safe, imagine how easy it would be to deploy for the average site.

Chris

4

(posting to -core as well.)

I know there has been a lot of talk about rails and thread safety.

Could you point me to these discussions?

Maybe I’ve missed it, but I havent’ seen any of the core team talk about whether they plan on fixing this in the near future or not. It seems to me that making rails thread safe would do more to enhance rails then any other single thing that could be done.

Why?

I’m not under any illusions, I’m sure it would be a lot of work. But I have to ask

myself the question. If it’s something that will have to be done at some point, and I think it will, doesn’t it make sense to tackle it sooner rather than later?

Yes, it does. I’m glad you’re pursuing it!

So what do the users think? Would you be willing to see work on other things stopped to get this fixed?

No. But handling concurrent requests within a single Ruby process would be wonderful.

Also, this isn’t a jab at the developers, I just haven’t seen anyone ask the question like this before. I’ve seen lots of discussion about how things are and particular areas in rails that are problematic, but no statements as to what might be done about it looking towards the

future or what the average rails user thinks about it.

And I will personally volunteer some of my time to help out if the decision is made to do something about this.

Thanks Chris. I’d start by removing the mutex in Mongrel and patching up any guts that spill out.

The greatest benefit I see is reduced memory footprint. By that standard, why aren’t the growing crop of shared Rails hosters pursuing it?

jeremy

5

Even without being thread safe the world isn't going to end, but if it were thread safe, imagine how easy it would be to deploy for the average site.

Would be great for VPS/shared hosting, yes. But probably not much of an issue for those who spend big bucks on rails..

Isak

6

> Maybe I've missed it, but I havent' seen any of the core team talk > about whether they plan on fixing this in the near future or not. It > seems to me that making rails thread safe would do more to enhance > rails then any other single thing that could be done.

Why?

Because then something like mongrel could just be a stand alone solution to hosting a lot of rails apps. No http proxy in front of it required nor would you need mongrel clusters. A lot of the complexity and learning curve for new users would disappear.

Thanks Chris. I'd start by removing the mutex in Mongrel and patching up any guts that spill out.

The greatest benefit I see is reduced memory footprint. By that standard, why aren't the growing crop of shared Rails hosters pursuing it?

hosting companies don't have an incentive. As long as everyone is on a level playing field and has to deal with it, what's the incentive of a single company to invest the time into making rails thread safe? Memory is a lot cheaper then man hours.

Like I said it's not the end of the world if rails isn't thread safe. It's just one of those things like ruby performance that I think is probably going to haunt rails until it's dealt with. That said, I also appreciate that the developers put their free time into creating rails, so my opinion is just and only that, an opinion.

7

Why not continuations instead?

8

We’ve resolved this under JRuby Rails deployments (mostly as proofs-of-concept) by having multiple instances of the JRuby runtime in-memory. It consumes a little more memory, but it allows us to scale Rails up to as many processors as are available while also simply having extra instances primed and ready for concurrent requests. We have not run any scalability tests, however, and Rails support under JRuby is still a bit of an adventure.

Ruby 2.0 is slated to have some sort of multi-VM capabilities like JRuby, where multiple independent instances of Ruby can co-exist within the same process. Also, whytheluckystiff has been working on sandboxing code that allows a similar feature under Ruby 1.8. However both cases are still in progress, and the JRuby option seems to be the most functional right now.

It’s also worth noting that other than some minimal memory savings, multi-threaded Rails wouldn’t really gain you much under Ruby 1.8 since threads are green in 1.8 (i.e. all Ruby threads are pumped by only one CPU thread). Using stock Ruby you really need to have multiple processes to scale properly.

9

I'm not quite as emphatic as snacktime when it comes to the importance of a thread-safe Rails, but it *would* do something to reduce the size of Mongrel packs (presumably), and (of more immediate importance to me) it would lift a big performance bottleneck in a Rails app I'm writing for an internal customer, where I need to query a bunch of web services and collate the info; having to do that single-threaded (because Rails requests are wrapped in a giant mutex) is killing response time for me. (Yes I've hacked around the problem, sorta, with forking, but it's so.... icky).

I know it's a pretty esoteric requirement, and not particularly broad-spectrum, but *I'd* certainly be interested in any effort to remove or reduce the thread-safety issues in Rails.

- Matt

10

Maybe I’ve missed it, but I havent’ seen any of the core team talk about whether they plan on fixing this in the near future or not. It seems to me that making rails thread safe would do more to enhance

rails then any other single thing that could be done.

Why?

Because then something like mongrel could just be a stand alone solution to hosting a lot of rails apps. No http proxy in front of it

required nor would you need mongrel clusters. A lot of the complexity and learning curve for new users would disappear.

Indeed - it’d also allow dev and production on the same platform. But why is this more important than any other single thing that could be done?

Note that thread safety doesn’t help much when the database doesn’t have a nonblocking API or the Ruby driver doesn’t support it.

Thanks Chris. I’d start by removing the mutex in Mongrel and patching up any guts that spill out.

The greatest benefit I see is reduced memory footprint. By that standard, why aren’t the growing crop of shared Rails hosters pursuing it?

hosting companies don’t have an incentive. As long as everyone is on a level playing field and has to deal with it, what’s the incentive of a single company to invest the time into making rails thread safe?

Memory is a lot cheaper then man hours.

This is true: your app gets a hit and has to be swapped in to serve it. Shared hosting is insane.

Like I said it’s not the end of the world if rails isn’t thread safe. It’s just one of those things like ruby performance that I think is probably going to haunt rails until it’s dealt with. That said, I also appreciate that the developers put their free time into creating

rails, so my opinion is just and only that, an opinion.

The cost/benefit just isn’t there yet, though a motivated hacker may find sufficient reward in improved code quality alone. Otherwise, it’s on the calendar pending Ruby2 + native threads.

As long as we’re slicing pie from the sky, how about _why-sandboxing in development mode, or having YARV spawn a VM per production app? Get mod_ruby back on the playing field…

Best, jeremy

11

Why?

I’m not quite as emphatic as snacktime when it comes to the importance of a thread-safe Rails, but it would do something to reduce the size of

Mongrel packs (presumably), and (of more immediate importance to me) it would lift a big performance bottleneck in a Rails app I’m writing for an internal customer, where I need to query a bunch of web services and

collate the info; having to do that single-threaded (because Rails requests are wrapped in a giant mutex) is killing response time for me. (Yes I’ve hacked around the problem, sorta, with forking, but it’s so… icky).

Birthing new mongrels in your pack is not an option? I think yours is the best case for better single-process concurrency - apps that aren’t hitting a db.

I know it’s a pretty esoteric requirement, and not particularly broad-spectrum, but I’d certainly be interested in any effort to remove or reduce the thread-safety issues in Rails.

Awesome! We have a handful of volunteers; let’s rub some code together. How about starting with a test suite for each component demonstrating where it isn’t thread-safe?

jeremy

12

It's also worth noting that other than some minimal memory savings, multi-threaded Rails wouldn't really gain you much under Ruby 1.8 since threads are green in 1.8 (i.e. all Ruby threads are pumped by only one CPU thread). Using stock Ruby you really need to have multiple processes to scale properly.

But that's only a small percentage of rails sites that even need duel cpu's. We have mod perl apps running on single cpu boxes that don't even breathe hard at 400,000 hits per day, and those servers are 2 years old. Fast drives and lots of memory though. Plus, even if things were thread safe you would still probably be running one process per application, so for hosting companies they would still be able to make use of more than one cpu.

One thing that definitely hasn't been tested is how much you could really gain by being able to run rails multithreaded, or where diminishing returns would start to kick in. For all I know that could happen at 3 threads or 20.

13

> > > Why? > > I'm not quite as emphatic as snacktime when it comes to the importance of > a thread-safe Rails, but it *would* do something to reduce the size of > Mongrel packs (presumably), and (of more immediate importance to me) it > would lift a big performance bottleneck in a Rails app I'm writing for > an internal customer, where I need to query a bunch of web services and > collate the info; having to do that single-threaded (because Rails > requests > are wrapped in a giant mutex) is killing response time for me. (Yes I've > hacked around the problem, sorta, with forking, but it's so.... icky).

Birthing new mongrels in your pack is not an option? I think yours is the

Not really, since the results of all of the different web services queries need to be aggregated into a single page and displayed to the user, all in real-time. I'm currently using forks to do the gathering in parallel, but it's not real good.

I know it's a pretty esoteric requirement, and not particularly > broad-spectrum, but *I'd* certainly be interested in any effort to remove > or > reduce the thread-safety issues in Rails.

Awesome! We have a handful of volunteers; let's rub some code together. How about starting with a test suite for each component demonstrating where it isn't thread-safe?

I think the suggestion to just pull the mutex and start hammering in some requests is a good start to at least get a feel for how much pain and suffering there's likely to be.

Unfortunately, to ensure that we've *really* gotten all the bugs, we'll need to audit all the code and verify that it's problem-free (or fix the problems). We could use the tightening noose method -- as we audit various parts of the code, the global mutex just wraps less and less of the code; eventually it should just wrap code that we *cannot* make thread-safe (external libraries, for instance) and user methods (we can't guarantee the thread-safety of that code, so it'll need to be kept on a leash).

Ultimately, though, the problem may require larger solutions -- changes in the coding practices of all contributors, or at least a constant vigilance on all committed code to ensure it's not introducing thread safety problems. I suspect that ultimately this might be the biggest hurdle to getting Rails thread-safe across the board. As Rasmus Lerdorf is fond of saying, "people aren't smart enough to write thread-safe code" -- it's excruciatingly difficult to get right.

Anyone up for Erlang on Rails? <grin>

- Matt

14

Jeremy Kemper wrote:

(posting to -core as well.)

I know there has been a lot of talk about rails and thread safety.

Could you point me to these discussions?

This, from Zed, is the most specific information I'm aware of:

   http://rubyforge.org/pipermail/mongrel-users/2006-May/000302.html

:frowning:

regards

   Justin

15

Rodrigo Rosauro wrote:

I think that making rails thread-safe would greatly expand its usability.

A 'thread' is like a 'goto'. It's useful in a pinch, but it's really bad structure.

Today, having background daemon threads is a pain in the azz because you need to start and control other ruby processes.

If you brought them closer to the Rails that serves pages, they would donate to it their fragility. They work best in other processes, with DRb between them and you, for insulation.

If you need to query them too often, then you probably have a latent time-sliced architecture. Converting that into something truly event-driven would probably improve its design, and make it leaner and more robust.

Next, threads and processes are a pain in the nuts to unit test. The blame doesn't belong to the tests. When you thread, and when you use semaphores to synchronize your threads, you commit the ultimate failure of encapsulation. You make one thread sensitive to variations in the tiniest private details of another thread's timing.

There are even some things completely impossible to do with rails, like as event-driven AJAX, or AJAX observer (like that GMail chat)... Which, personally, I think is one of the big deals of ajax to deliver instantaneous responses...

I currently use BackgrounDRb in two projects. For both of them, I first tried the non-detached version. They both didn't work. One, zipping up a bunch of files, took too long. So I let the user click, start a task, start a spinning graphic on the user interface, and wait for the task to end. A periodically_call_remote polls the task. When it finishes, the remote call returns JavaScript that sets a small IFRAME's src attribute to the server location of the zipped file. That pops up the Save File dialog automatically.

For the other project, a game, I need abstract timing events to update different web browsers simultaneously, in real-time. Again, I first tried that without a background process, and it was too slow and clumsy. So I put the event timer, and its state table, into a BackgrounDRb process, and I use Juggernaut to send new JavaScript updates to both browsers.

GMail is similar. They hacked their server to leave a TCP channel open, providing "server push". Don't do that if you are not Google, and don't have a horde of engineers to hack your server. Juggernaut is a nice compromise if you already use Flash. It uses Flash to leave that wire open, enabling server-push.

And never think for a second that a combination of BackgrounDRb and traditional Ajax will be more responsive than old-school Ajax. Speaking from personal experience, you will get complaints from your ISP that you are using too much CPU time on their server!