A way to share a variable across multiple httpd threads with Rails?

Let's say I have the following threading in my Rails web application:

    class MyController
      def my_action
        count = 0
        arr = []

        10.times do |i|
          arr[i] = Thread.new {
            sleep(rand(0)/10.0)
            Thread.current["mycount"] = count
            count += 1
          }
        end

        arr.each {|t| t.join; print t["mycount"], ", " }
        puts "count = #{count}"
      end
    end

As you can see, the 'count' variable is shared across all threads.

**Now,** what I want to do is **share 'count'** across multiple httpd
requests and **allow my_action in MyController to have access to that
variable**. For instance, maybe whatever spawns the ruby process to
serve httpd process could hold the variable count in its scope, and then
the ruby processes spawned for httpd processes could then access that
variable.

Using memcached, a database, and session variables is out of the
question. Ultimately 'count' will actually be a resource object...an FTP
connection.

Is this possible? Perhaps using Apache/Passenger workers like at
http://stackoverflow.com/questions/4471680/a-way-to-access-common-ftp-connection-resource-pool-in-ruby-across-ajax-calls/4471758#4471758?

Example code would be appreciated.

Let's say I have the following threading in my Rails web application:

   class MyController
     def my_action
       count = 0
       arr = []

       10.times do |i|
         arr[i] = Thread.new {
           sleep(rand(0)/10.0)
           Thread.current["mycount"] = count
           count += 1
         }
       end

This looks thread dangerous to me - i don't think += is atomic.

       arr.each {|t| t.join; print t["mycount"], ", " }
       puts "count = #{count}"
     end
   end

As you can see, the 'count' variable is shared across all threads.

**Now,** what I want to do is **share 'count'** across multiple httpd
requests and **allow my_action in MyController to have access to that
variable**. For instance, maybe whatever spawns the ruby process to
serve httpd process could hold the variable count in its scope, and then
the ruby processes spawned for httpd processes could then access that
variable.

Using memcached, a database, and session variables is out of the
question. Ultimately 'count' will actually be a resource object...an FTP
connection.

Is this possible?

I wouldn't go down this road if I were you. For starters presumably the objects in question will be modified by those workers, you'll need to synchronise access to them etc. You might also be painting yourself into a corner for the day you need to scale beyond 1 server.

Personally, if I had to do this, I'd these objects in a separate long-lived process (or processes) and have your controller actions talk to these processes (exactly how depends on what you'll be doing, you might consider drb, message queues etc)

Fred

Frederick Cheung wrote in post #986539:

           Thread.current["mycount"] = count
           count += 1
         }
       end

This looks thread dangerous to me - i don't think += is atomic.

variable**. For instance, maybe whatever spawns the ruby process to
serve httpd process could hold the variable count in its scope, and then
the ruby processes spawned for httpd processes could then access that
variable.

Just example code I pulled from someone online. I'd never actually use
this, and my stuff would be as thread-safe as possible.

Personally, if I had to do this, I'd these objects in a separate
long-lived process (or processes) and have your controller actions talk
to these processes (exactly how depends on what you'll be doing, you
might consider drb, message queues etc)

You're talking about something like a daemon to act as a proxy, and
I'd communicate with it via some protocol I define? That was my thought
too, but it's extra work, and I was trying to avoid spending the extra
time on this and just use the FTP connection directly.

So, regardless of whether this is a good or bad idea (I will assess that
later), is there any way to directly access a shared resource object?

So, regardless of whether this is a good or bad idea (I will assess that
later), is there any way to directly access a shared resource object?

Regardless of that: I think that passenger has a callback that is
executed before new worker instance is spawned. Maybe that could
somehow help you.

Robert Pankowecki

Frederick Cheung wrote in post #986539:

          Thread.current["mycount"] = count
          count += 1
        }
      end

This looks thread dangerous to me - i don't think += is atomic.

variable**. For instance, maybe whatever spawns the ruby process to
serve httpd process could hold the variable count in its scope, and then
the ruby processes spawned for httpd processes could then access that
variable.

Just example code I pulled from someone online. I'd never actually use
this, and my stuff would be as thread-safe as possible.

Personally, if I had to do this, I'd these objects in a separate
long-lived process (or processes) and have your controller actions talk
to these processes (exactly how depends on what you'll be doing, you
might consider drb, message queues etc)

You're talking about something like a daemon to act as a proxy, and
I'd communicate with it via some protocol I define? That was my thought
too, but it's extra work, and I was trying to avoid spending the extra
time on this and just use the FTP connection directly.

I think you'd find that drb is very easy to use.

So, regardless of whether this is a good or bad idea (I will assess that
later), is there any way to directly access a shared resource object?

I doubt it - while with something like passenger your workers do share initial resources (since they are forked from the application spawner (at least by default)), that's a one way thing: if one worker changes the state of the object then that happens in that process only. You do share file descriptors, but that's more likely to be a nuisance - if two workers try use the same socket as the same time, chaos will ensue

Fred

I don't think there's any *question* this is a bad idea. For instance,
how are you planning on arbitrating the FTP connections to avoid
situations like this:

Process 1: cd foo
Process 2: cd bar
Process 1: put somefile.txt

Now somefile.txt is in the wrong place... There are similar issues
with keeping the connections alive, etc plus the *total* mess that
happens once you move beyond a single box.

In the Stackoverflow question, you mention that you're only listing
directories. Maybe it would make more sense to write a fairly basic
proxy in a smaller framework (Sinatra maybe?) that just takes JSON
specifying what to list and spits back results? Such a server would
solve the issues mentioned above, and could even do basic caching.

--Matt Jones