I am looking for the most direct approach to launching ansyncronous database writes. These would be one-way, fire & forget. I can tolerate the occasional lost message.
Been doing a bunch of reading on async process for Rails. I see several "process schedulers" that have been written such as BackgroundJob, AP4R, etc. (saw the thread re: forking/threading).
I'm fairly certain I don't want these types of systems. I don't need a "scheduler," I just want to send some writes to a separate database during the course of processing a Rails request and not have the Rails processes held up by those connections.
In my previous work with Lasso, all that was needed for that was adding an -async option to a method, and voila, a completely independent and async thread was launched for that code that took advantage of additional CPUs, etc.
From what I gather about Ruby, I think all I want to do is use Thread.new with a raw mysql-ruby connection (don't need ActiveRecord for this) in the block. Does that sound about right?
And, just for curiousity sake, it would appear that launching a new Thread in Ruby would actually remain tied to the same CPU core? This is the "family secret" about threads that one reads about Ruby? So conceivably if the main thread or the new thread were intense enough, launching a new thread wouldn't actually provide any practical concurrent gains? I know that I most cases gains are probably made, just asking if that's the talking point some people have against Ruby & threading.
In my previous work with Lasso, all that was needed for that was
adding an -async option to a method, and voila, a completely
independent and async thread was launched for that code that took
advantage of additional CPUs, etc.
From what I gather about Ruby, I think all I want to do is use
Thread.new with a raw mysql-ruby connection (don't need ActiveRecord
for this) in the block. Does that sound about right?
And, just for curiousity sake, it would appear that launching a new
Thread in Ruby would actually remain tied to the same CPU core? This
is the "family secret" about threads that one reads about Ruby?
MRI's threads are so called green threads - they exist entirely within
the interpreter, from the point of the OS there is only ever a single
thread (and so only one ruby thread actually ever runs at once). This
approach to threading also means a single ruby thread can block the
entire ruby vm.
This won't happen with pure ruby code, since ruby's scheduler takes
care of that but can happen with c extensions (so to give the simplest
example, a c extension that just calls sleep(5) will block all ruby
threads for 5 seconds). So if (for example) the c interface to mysql
were to be blocking then a mysql query would block the app, threads or
no threads. Unfortunately that is the case.
The pure ruby mysql doesn't suffer from that problem, however it's not
recommended for production use (and you can't mix and match them
inside a single ruby process.
It's also the case that jruby gives you 'real' threads.
Fred
Thanks for the description.
So Thread.new might allow Ruby to work on two pure Ruby workloads
concurrently (two big iterates or something), but the scenario I was
after is pretty much just going to be linear anyway? Bummer.
