Does Rails persist in RAM between requests?

rubyonrails-talk
1

I'm trying to implement a data caching system and getting nowhere because I cannot get a ruby global variable to persist across page requests.*

This leads me to really wonder how the entire process of mongrel- >ruby->rails is working.

If I run a plain Ruby shell script, it starts, it ends, done. Processing the file is the extent of the life cycle of the code. I get that.

Is Rails essentially just a giant nested set of scripts with a single start and finish life cycle as well?

Does Rails persist as an in-memory application even when not responding to requests? If so, I am totally baffled by global variables being re-initialized with each page request.

Perhaps understanding this process will help me see if what I am trying to is even supposed to work or not.

-- gw

* How make persistent in-memory objects? - Rails - Ruby-Forum

2

I'm trying to implement a data caching system and getting nowhere because I cannot get a ruby global variable to persist across page requests.*

Can you look into how the session code works?

Is Rails essentially just a giant nested set of scripts with a single start and finish life cycle as well?

Sort of.

Does Rails persist as an in-memory application even when not responding to requests? If so, I am totally baffled by global variables being re-initialized with each page request.

It depends.

For instance, web servers can be configured to run at most N requests through a single process, then let it die. This is a common strategy to "fixing" memory leaks.

So, your application might restart fully, or it might just reset a lot of state. You can't really rely on either behavior.

In development mode, it doesn't always RELOAD everything, it just checks the timestamps on a lot more files. For instance, in production I've found view templates will automatically reload, probably because they are actually re-read from disk each time. Controllers and plug-ins will not, nor will any configuration files. In development mode, controllers will reload if changed, but most configuration will not.

--Michael

3

I'm trying to implement a data caching system and getting nowhere because I cannot get a ruby global variable to persist across page requests.*

This leads me to really wonder how the entire process of mongrel- >ruby->rails is working.

If I run a plain Ruby shell script, it starts, it ends, done. Processing the file is the extent of the life cycle of the code. I get that.

Is Rails essentially just a giant nested set of scripts with a single start and finish life cycle as well?

No.

Does Rails persist as an in-memory application even when not responding to requests? If so, I am totally baffled by global variables being re-initialized with each page request.

Yes.

Are you running in development mode? Development mode reloads an awful lot b/n each request and might be undoing whatever you're trying to persist.

-philip

4

Does Rails persist as an in-memory application even when not responding to requests? If so, I am totally baffled by global variables being re-initialized with each page request.

Yes.

OK, at least that point is settled. I figured it had to be, but...

Are you running in development mode? Development mode reloads an awful lot b/n each request and might be undoing whatever you're trying to persist.

Yep. Hmm. Y'think it would even wipe out global vars? I suppose it could -- I had a mechanism in my own framework ('nother language) to do that very thing, but I could control turn it on/off at will rater than tying it exclusively to a mode. Hmm... OK, I'll experiment with running is a production mode I gueess.

Thanks! At least it is something new to try.

-- gw

5

Not likely dev mode is dorking around with initialization stuff. It mostly tries to reload models to reflect code changes. Try this. Start a new mongrel on another port so you can be sure you have a real fresh copy with real fresh code. Put logging statements in your caching code to determine the state of the variable you're trying to access. I always use RAILS_DEFAULT_LOGGER.debug but do as you see fit.

Tail your development.log to see what that reveals. If you are totally stuck, try ruby-debug (http://www.datanoise.com/articles/2006/07/12/tutorial-on-ruby-debug) and set a breakpoint.

G'luck.

6

This yields some differences. Still some screwy behavior at times, but getting further. Whatever is going on it seems to be some condition induced by my attempts to stop/start everything maybe?

OK, I'll hack at it for a while longer with these ideas.

Thanks for the fresh angle.

-- gw

7

Have you tried to run it in production mode to see how the behavior changes?

Pat

8

Finally got somewhere with this one. Starting a new port generate some different results. Still not perfect, but at least different. So, I took that thoiughtto the extreme and just rebooted the whole machine. Now it works.

Very freaky that two things didn't work (global var across pages, and added a var to the environment page), but everything else did work. Wasted a lot of time on this one. I hope its not an omen!

Thanks everyone for poking at it with me. Hopefully tomorrow I can make some real progress (the alleged Rails productivity meter is a bit in the yellow zone after today :wink:

-- gw

9

Do consider "memcached".

Regards, Isak

10

> > I'm trying to implement a data caching system and getting nowhere > because I cannot get a ruby global variable to persist across page > requests.* >

Do consider "memcached".

Never mind, I see from one of your other topics on this issue that you're already aware of memcached.

Data in a global/singleton class should persist fine across requests to the same rails instance, but if you need to share the cache between multiple rails processes, memcached is the way to go.

You could also use the file system, relying on the OS to keep your data cached in memory, but that sounds awkward at best..

11

I will, but I just didn't need the added complexity _today_* :slight_smile:

I did get my global var cache working, but unlike the prev environment I worked with, I can see where this idea will be kinda useless in Rails as each instance in production will be maintaining a redundant copy of what I store.

However, I'm trying to learn a bunch of new stuff, so just did not need to branch into memcached today. One thing about memecached on Rails is that I don't see how to use it for arbitrary data chunks. Rails is obsessed with caching HTML, but I don't see any provisions for arbitrary data. I have stuff I want to load (from txt, db, even web service) and pre-parse/munge it to a particular state, store it, retrieve it in that state and finish the job based on other application logic influences.

For that, I am accustomed to building my own RAM caching via globals, but that's in a laguage where the environment is truly global unlike it would be in a mulit-process mongrel/Rails setup.

So I see memcached being in my future soon enough, but need more time to read up on it.

Meanwhile I can move dev forward with my cheesy home-grown one for a while.

* I've been using Lasso for years now. It's totally turn-key. All this stuff I see the Rails world cobbling together out of pieces of open-source pet projects, and spending who knows how many days and weeks it take to learn how to put it together, Lasso just has out of the box, ready to go. It's not perfect, but edags the setup and hosting and all that is so much easier, and optimized for web apps. Oh, well.

-- gw

12

Regards, Isak

13

This isn't data that belongs in a session.

I use globals as a mini memcache (at least I did with Lasso, and now I'm assuming I can recreate that with Ruby). It's a hash of data being stored in partially processed formats.

In my case I'll have a particular global dedicated to caching a particular type of data. The global will be created by the system that needs it, when it needs it. This way I can build sub-systems that create their own environment needs. If I need a db mapper, I install that. It creates it's own global for caching the various cross reference structures created from mapping definitions.

Example: I load value list definitions from a text file (or db, or even web service, source doesn't matter). Many value lists are stored in one file, or retrieved with one query. These definitions do not change, they're part of fixed options in the application design. I lazy load the lists as they're needed, parse each list into a data structure suitable for passing as options to a renderer, then cache each parsed list.

I have my own value list renderer class that provides some options and application-integration capabilities different than the Rails one. Some lists require on the fly additions or subtractions at various points in the application (modular app hierarchy, or permissions based adjustments). I can pull the base list from the cache and modify it in the controller before passing a var with the final list to the renderer in the view.

Also, I might go ahead and store a baseline HTML rendering of a value list in the cache, and then use regex to alter its final state. This is usually the fastest way to generate valuelists. On pages with a handful of them it doesn't matter so much. On pages with many of them or, several long ones, it can make a difference.

I know Rails has some tools for this, but IMO they're more cumbersome than my system, and they simply don't allow for some of the flexibility I need.

My using globals tends to be for application config/reference, so while using globals is good for creating machine-specific caches, I wouldn't use it for data where it would make more sense to use something like memcached.

-- gw

14

Sure, but in the context of Rails, Rails' own cache APIs all seem to be geared towards caching HTML or database queries. So if I generate this chunk of data internal to my app (whether it is just text or an object that could be serialized), I don't see any Rails APIs for, say, memecached that are for storing that particular chunk. From what I can tell so far, Rails integration of memcached is as a sub-service to its own higher level APIs for storing HTML and db queries.

So, there has to be some way to talk to memcached directly so I can build my own higher level API for storing arbitrary data. I'm sure it's there somewhere, but I don't see it documented clearly... yet.

-- gw

15

http://dev.robotcoop.com/Libraries/memcache-client/index.html

Download the memcache_util.rb file and look at it.

Once loaded you can do things like:

CACHE.set("key", "val") CACHE.get("key")

etc...

And you're talking directly to memcache... no rails stuff..

-philip

16

OK, cool. Does this replace the interface already in Rails? Or work in parallel?

I will get to it, but probably not for a while. Trying to fry other fish first.

-- gw

17

So, there has to be some way to talk to memcached directly so I can build my own higher level API for storing arbitrary data. I'm sure it's there somewhere, but I don't see it documented clearly... yet.

http://dev.robotcoop.com/Libraries/memcache-client/index.html Download the memcache_util.rb file and look at it. Once loaded you can do things like: CACHE.set("key", "val") CACHE.get("key") And you're talking directly to memcache... no rails stuff..

OK, cool. Does this replace the interface already in Rails? Or work in parallel?

Doesn't interfer with Rails at all. memcache-client could be happily used in pure ruby land. Unless you configure the other memcache related things, you won't get any memcache auto-magic in rails.

18

Good. sounds like a better place to start for me -- work w/ it directly, then figure out the Rails implementation if I need it.

Thx.

-- gw

19

Hi Greg

20

Understood, but in this case, that's OK because the global isn't really being used for the purposes of sharing, it is being used as a high speed replacement for text files or db. The original data is more or less static. It gets loaded mostly from text files, sometimes a database. Loaded once, parsed, then used for repetitive reads. It doesn't change.

We're talking about a relatively small amount of data--50 to 100 KB max. So, each process instance can create it's own global, that's not a problem.

-- gw