memcache-client extensions plugin

rubyonrails-talk
1

From: http://blog.zvents.com/2006/11/1/rails-plugin-memcacheclient-extensions

=MemCacheClient Extensions

== About

The memcache-client_extensions plugins adds three new commands to the memcache client API:

1. get_multi : retrieve more than 1 key in parallel 2. stats : retrieve server performance and utilization statistics 3. flush_all : empty all information stored in memcached

== Installation

1. This plugin requires that the memcache-client gem is installed.    # gem install memcache-client

2. Install the plugin or the gem    $ script/plugin install svn://rubyforge.org/var/svn/zventstools/projects/memcache-client_extensions - OR -    # gem install memcache-client_extensions

== get_multi

Retrieve multiple values from memcached in parallel, if possible. The memcached protocol supports the ability to retrieve multiple keys in a single request. Pass in an array of keys to this method and it will:   a. map the key to the appropriate memcached server   b. send a single request to each server that has one or more key values

Returns a hash of values.

CACHE["a"] = 1

=> 1

CACHE["b"] = 2

=> 2

CACHE.get_multi(["a","b"])

=> {"a"=>1, "b"=>2}

Here's a benchmark showing the speedup:

CACHE["a"] = 1 CACHE["b"] = 2 CACHE["c"] = 3 CACHE["d"] = 4 keys = ["a","b","c","d","e"] Benchmark.bm do |x|   x.report { for i in 1..1000; keys.each{|k| CACHE.get(k);} end }   x.report { for i in 1..1000; CACHE.get_multi(keys); end } end

returns:      user system total real 0.180000 0.130000 0.310000 ( 0.459418) 0.200000 0.030000 0.230000 ( 0.269632)

There's a fair amount of non-DRY between get_multi and get (and threadsafe_cache_get/multi_threadsafe_cache_get and cache_get/multi_cache_get for that matter) but I think it's worth it since the extra overhead to handle multiple return values is unneeded for a single-key get (which is by far the most common case).

== stats

The stats method returns statistics for each memcached server. An explanation of the statistics can be found in the memcached docs: http://cvs.danga.com/browse.cgi/wcmtools/memcached/doc/protocol.txt?rev=HEAD&content-type=text/plain

Example:

CACHE.stats

=> {"localhost:11211"=>{"pid"=>"20188", "bytes"=>"4718", "connection_structures"=>"4", "time"=>"1162278121", "pointer_size"=>"32", "limit_maxbytes"=>"67108864", "version"=>"1.2.0", "cmd_get"=>"14532", "cmd_set"=>"32", "bytes_written"=>"432583", "uptime"=>"1557", "curr_items"=>"4", "curr_connections"=>"3", "total_connections"=>"19", "get_misses"=>"0", "rusage_user"=>"0.119981", "rusage_system"=>"0.313952", "total_items"=>"32", "get_hits"=>"14532", "bytes_read"=>"190619"}}

== flush_all

The flush_all method empties all cache namespaces on all memcached servers. This method is very useful for testing your code with memcached since you normally want to reset the cache to a known (empty) state at the beginning of each test.

== Bugs, Code and Contributing

There's a RubyForge project set up at:

http://rubyforge.org/projects/zventstools/

Anonymous SVN access:

$ svn checkout svn://rubyforge.org/var/svn/zventstools

Author: Tyler Kovacs (tyler dot kovacs at gmail dot com)

2

Cool :slight_smile:

Slightly OT here, but is there a way to access all values from a namespace in memcached? Like “get all my books” or something. Seems like a good extension to your get_multi function.

But perhaps the memcached protocol doesn’t allow for this. That was my finding the last time I read through it. Somebody should fix this though: clearing/adding-to/retrieving a namespace is a real good feature to add.

Vish

3

No, the memcached protocol doesn't support a "get all" method. Here's a thread from the memcached mailing that covers the issue:

http://lists.danga.com/pipermail/memcached/2006-April/002112.html

You could memcache your dataset in an array (in addition to memcaching each individual item for single-item retrieval), but that probably wouldn't be a good idea for anything other than very small datasets.

Vishnu Gopal wrote: