Caching of data on server?

Hi

I am looking to speed up my rails application and am working through all the SQL queries and trying to remove unnecessary queries.

I thought of the following and would love to hear whether it is possible.

I have one (smallish - about 40 records) table which only gets read by my app - no writing. It gets read a lot though. Would it be possible to preload this table into memory (as a kind of global global dataset) when the server starts up? This would save me at least one sql query on every server trip.

Is this even necessary? Maybe rails - (or the database?) automatically caches things like this?

Your thoughts and suggestions are most welcome.

Pieter Hugo

Hi Hugo,

Have a look at Rails caching. Here is a tutorial

All the best,

Fidel.

Hi Pieter,

Rails does some class caching in the default production environment. There's more you can do though. It depends on what kind of website you have. If you only serve semi-static content, caching your pages (or parts of it) will improve your speed incredibly. No queries will be made until something's updated (mind that you will have to expire certain cache elements on changes in the database).

If you do a lot of searching, caching the pages will probably not work for you or only do the trick partly. Do you use indexing in your database? Using indexes can make database actions much faster.

Kind regards, Jaap Haagmans

I have done something similar with a small data set that was being referenced a lot. In the end however the improvement, whilst noticeable, was not that significant.

Does this data set used as it is or is it transformed before use?

If so it might make sense to cache the transformed data or even the pages as as been suggested.

Hi Guys

Thanks for the ideas. Some good stuff here to look into.

After thinking about this a bit more I'd like to slightly rephrase my question:

Is it possible to create variables in the rails environment that persist globally - not as sessions - that would be too much porting of information forwards and back. In any case the data needs to be visible 'between' sessions as well.

In old style Visual Basic ASP there was a file called global.asa where one could define variables that were always available to all sessions. Is there a similar thing in rails?

Thanks for the pointers once again.

Hi Pieter

Is it possible to create variables in the rails environment that persist globally - not as sessions - that would be too much porting of information forwards and back.

   This is possible by global variables in ruby (variables starts with @@ )

In any case the data needs to be visible 'between' sessions as well.

   But this needs the data to be stored in D B it self. Is it?

Sijo

Hi Sijo

Thanks for the info - I will look in to the @@ idea

In any case the data needs to be visible 'between' sessions as well.

   But this needs the data to be stored in D B it self. Is it?

Yes - The data is currently in a table in the database - it is a table of rights users have under different conditions. These rights wont change for weeks on end, so reading them from the database every time the user tries to do something is silly. I want these rights to be kept in the memory of the server all the time. In other words - when the web server is started - get the rights from the database and keep them in memory forever (or until the server is restarted) Whenever a request comes in from a user - check the right against memory instead of against the database.

Is the environment.rb and the production/development/test files executed every time a request comes in or only when the server starts up? If it is the latter I should be able to create variables there that are visible 'across' session?

Regards

Pieter

I would not go for this sort of optimisation unless you know that you have a performance issue. It may seem unnecessary to access the db each time but the data will be cached by the db server so the overhead should be minimal. It is most unlikely that this will turn out to be a significant part of your request turnaround time.

Colin

Sijo k g wrote:

Hi Pieter

Is it possible to create variables in the rails environment that persist globally - not as sessions - that would be too much porting of information forwards and back.

   This is possible by global variables in ruby (variables starts with @@ )

No, those are class variables. Global variable names start with different characters in Ruby. If you really want to know what, you can look it up, but I'm not going to make it easier to implement a bad idea.

And global variables *are* a bad idea. In Rails, they're not even reliable for maintaining global state, because they're only global to each server instance. Just forget they exist.

In any case the data needs to be visible 'between' sessions as well.

   But this needs the data to be stored in D B it self. Is it?

Sure sounds like it.

Sijo

Best,

Marnen Laibow-Koser wrote:

Sijo k g wrote:

Hi Pieter

No, those are class variables. Global variable names start with different characters in Ruby.

O Sorry That was my mistake. Global variables starts with $. Example $var. And you can take a loook at

Sijo

Hi Guys

Thanks for the help - and the warning. I got the global variable thing working - in principle - but now rails complains that I cannot do something like

$rights = Rights.find(:all)

in environment.rb

I'm giving it up for a bad job - (not that I need to squeeze the last drop of performance out of the system yet) I think I'll stick to reading from the db every time as you suggest. (There is probably already caching of the table going on in the background in any case)

I take the point that global variables are not stable enough, but it still strikes me as wasteful having to constantly access a small table which I know is not changing - millions of times more disk access and processor cycles than just reading it when the web server starts up and tagging it along in memory.

But then - I am the first to admit that I am not really qualified to be making these statements :wink:

Thanks again for the assistance. You guys are great!

Pieter