Asp Forum - [ANN] Localmemcache-0.2.0: The beauty of memcached. For local data. Blazingly fast.

Sven C. Koehler

3/30/2009 11:18:00 PM

Hi,

I am happy to announce the next version of localmemcache (0.2.0), a
library that aims to provide an interface similar to memcached but for
accessing local data instead of remote data. It's based on mmap()'ed
shared memory for maximum speed.

* http://localmemcache.ruby...

Changes for 0.2.0:
* Localmemcache uses logging now: In case your application is terminated
while accessing the shared memory (eg by a kill -9), localmemcache is
now able to restore the integrity of your data.
* \0 character can now be used in keys and values
* The ruby binding now features a keys() method.
* Added a C API.

0.2.0 facts:
- lightweight: The core library is just about 1300 lines of C code
- fast: About 40% slower than Ruby's hash (or about 36 times
faster than a local memcached).

EXAMPLE
=======

require 'localmemcache'
$lm = LocalMemCache.new :namespace => "viewcounters"
$lm[:foo] = 1
$lm[:foo]
$lm.delete(:foo)

INSTALL
=======

# gem install localmemcache

(In case rubyforge has not yet updated the mirrors, fetch the 0.2.0 gem
from here: http://github.com/sck/localmemcache... and then do
# gem install localmemcache-0.2.0.gem )

CONTACT
=======

Please contact me with bugs, suggestions and patches at: schween + snafu # de

LINKS
=====

Localmemcache: http://localmemcache.ruby...
Rubyforge project: http://localmemcache.ruby...

Source code is hosted on github: http://github.com/sck/loca...

Best,

Sven C. Koehler

afa5fe7a3a23930a50d0b8b4a1a55880 localmemcache-0.2.0.gem
050cab44c9a5b9cb6214779398c0cfb7 localmemcache-0.2.0.tar.gz

12 Answers

ara.t.howard

3/31/2009 2:27:00 AM

On Mar 30, 2009, at 5:17 PM, Sven C. Koehler wrote:

> # gem install localmemcache

really nice idea. can't wait to try it out!

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Bill Kelly

3/31/2009 7:14:00 AM

From: "Sven C. Koehler" <schween@snafu.de>
>
> I am happy to announce the next version of localmemcache (0.2.0), a
> library that aims to provide an interface similar to memcached but for
> accessing local data instead of remote data. It's based on mmap()'ed
> shared memory for maximum speed.

Very nice!!

I've been daydreaming about a mmap'd hash table for a
couple years now -- and I wasn't even contemplating
having it able to be shared between processes, as it
appears you've done.

Your code appears nicely decoupled. It seems possible
if one were to write Windoze versions of lmc_lock.c and
lmc_shm.c , the rest of the code might just work on that
platform. (I may attempt this... someday...)

Anyway, thanks for releasing this interesting library!

Regards,

Bill

Sven C. Koehler

3/31/2009 2:33:00 PM

On Tue, Mar 31, 2009 at 04:13:39PM +0900, Bill Kelly wrote:
> From: "Sven C. Koehler" <schween@snafu.de>
>>
>> I am happy to announce the next version of localmemcache (0.2.0), a
>> library that aims to provide an interface similar to memcached but for
>> accessing local data instead of remote data. It's based on mmap()'ed
>> shared memory for maximum speed.
>
> Very nice!!

Thanks!

> Your code appears nicely decoupled. It seems possible
> if one were to write Windoze versions of lmc_lock.c and
> lmc_shm.c , the rest of the code might just work on that
> platform. (I may attempt this... someday...)

Yeah, this should work. ;-)

-S.

Michael Fellinger

4/4/2009 4:36:00 PM

On Tue, 31 Mar 2009 08:17:45 +0900
"Sven C. Koehler" <schween@snafu.de> wrote:

> Hi,
>
> I am happy to announce the next version of localmemcache (0.2.0), a
> library that aims to provide an interface similar to memcached but for
> accessing local data instead of remote data. It's based on mmap()'ed
> shared memory for maximum speed.

This sounds great, can't wait to add it as a caching option to Ramaze.

> * http://localmemcache.ruby...
>
> Changes for 0.2.0:
> * Localmemcache uses logging now: In case your application is
> terminated while accessing the shared memory (eg by a kill -9),
> localmemcache is now able to restore the integrity of your data.
> * \0 character can now be used in keys and values
> * The ruby binding now features a keys() method.
> * Added a C API.
>
> 0.2.0 facts:
> - lightweight: The core library is just about 1300 lines of C code
> - fast: About 40% slower than Ruby's hash (or about 36 times
> faster than a local memcached).
>
> EXAMPLE
> =======
>
> require 'localmemcache'
> $lm = LocalMemCache.new :namespace => "viewcounters"
> $lm[:foo] = 1
> $lm[:foo]
> $lm.delete(:foo)

I tried that, but $lm[:foo] always returned nil.
That's on ArchLinux, both 32 and 64bit.

--
^ manveru

Radoslaw Bulat

4/4/2009 4:50:00 PM

On Sat, Apr 4, 2009 at 6:36 PM, Michael Fellinger <m.fellinger@gmail.com> w=
rote:
>> require 'localmemcache'
>> $lm =3D LocalMemCache.new :namespace =3D> "viewcounters"
>> $lm[:foo] =3D 1
>> $lm[:foo]
>> $lm.delete(:foo)
>
> I tried that, but $lm[:foo] always returned nil.
> That's on ArchLinux, both 32 and 64bit.
>

The same for me. My configuration: ubuntu 8.10 64bit, ruby 1.8.7.

--=20
Pozdrawiam

Rados=B3aw Bu=B3at
http://radarek... - m=F3j blog

Sven C. Koehler

4/4/2009 5:51:00 PM

On Sun, Apr 05, 2009 at 01:36:09AM +0900, Michael Fellinger wrote:
> On Tue, 31 Mar 2009 08:17:45 +0900
> "Sven C. Koehler" <schween@snafu.de> wrote:
> > require 'localmemcache'
> > $lm = LocalMemCache.new :namespace => "viewcounters"
> > $lm[:foo] = 1
> > $lm[:foo]
> > $lm.delete(:foo)
>
> I tried that, but $lm[:foo] always returned nil.
> That's on ArchLinux, both 32 and 64bit.

Hmm... Can you see the namespace in /var/tmp/localmemcache/ ?
Can you see the key in $lm.keys()?
Does ``du -h /var/tmp/localmemcache/viewcounters.lmc'' show that there's
content in the file?

Best,

Sven

Sven C. Koehler

4/4/2009 6:00:00 PM

Radoslaw Bulat

4/4/2009 9:57:00 PM

On Sat, Apr 4, 2009 at 7:50 PM, Sven C. Koehler <schween@snafu.de> wrote:
> Hmm... Can you see the namespace in /var/tmp/localmemcache/ ?

Yes
$ ls /var/tmp/localmemcache/
foo.lmc viewcounters.lmc

> Can you see the key in $lm.keys()?
Yes, but they are duplicated after setting on the same key:

>> $lm =3D LocalMemCache.new :namespace =3D> "viewcounters"
=3D> #<LocalMemCache:0x7f9c05d844d0>
>> $lm.keys
=3D> []
>> $lm[:foo] =3D 1
=3D> 1
>> $lm[:foo]
=3D> nil
>> $lm.keys
=3D> ["foo"]
>> $lm[:foo] =3D 2
=3D> 2
>> $lm[:foo]
=3D> nil
>> $lm.keys
=3D> ["foo", "foo"]

> Does ``du -h /var/tmp/localmemcache/viewcounters.lmc'' show that there's
> content in the file?
>

$ du -h /var/tmp/localmemcache/viewcounters.lmc
20K /var/tmp/localmemcache/viewcounters.lmc

--=20
Pozdrawiam

Rados=B3aw Bu=B3at
http://radarek... - m=F3j blog

Matthias Reitinger

4/4/2009 10:05:00 PM

On Sat, 04 Apr 2009 12:59:56 -0500, Sven C. Koehler wrote:
> This is how it looks on my machine:

I can reproduce the issue Michael and RadosÅ?aw are having:

reima@marvin:~$ uname -a
Linux marvin 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686
GNU/Linux
reima@marvin:~$ ruby -v
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
reima@marvin:~$ irb
>> require 'localmemcache'
=> true
>> $lm = LocalMemCache.new :namespace => "viewcounters"
=> #<LocalMemCache:0xb7b69bdc>
>> $lm.keys
=> []
>> $lm[:foo]
=> nil
>> $lm[:foo] = 1
=> 1
>> $lm[:foo]
=> nil
>> $lm.keys
=> ["foo"]
>> puts `du -h /var/tmp/localmemcache/viewcounters.lmc`
16K /var/tmp/localmemcache/viewcounters.lmc
=> nil
>>

And on a sidenote:

reima@marvin:~$ irb
>> require 'localmemcache'
=> true
>> $lm = LocalMemCache.new :namespace => "viewcounters"
=> #<LocalMemCache:0xb79eac20>
>> $lm.close
=> nil
>> $lm.keys
(irb):4: [BUG] Segmentation fault
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]

Aborted
reima@marvin:~$

-Matthias

Sven C. Koehler

4/5/2009 1:09:00 PM

Ok, thanks to everyone who reported bugs!
I think I fixed this problem, so could anyone who had problems setting
keys run the following command to verify whether it works better now?

% git clone git://github.com/sck/localmemcache.git && cd localmemcache && rake sanity_test

The output at the end should look like this:

| LocalMemCache
| - should allow to set and query keys
| - should support the [] and []= operators
| - should allow deletion of keys
| - should return a list of keys
| - should support \0 in values and keys
| - should throw exception if pool is full
| - should support checking of namespaces
| - should support clearing of namespaces
|
| 8 specifications (8 requirements), 0 failures, 0 errors

(I haven't fixed the close bug yet.)

Best,

Sven

On Sun, Apr 05, 2009 at 07:09:56AM +0900, Matthias Reitinger wrote:
> On Sat, 04 Apr 2009 12:59:56 -0500, Sven C. Koehler wrote:
> > This is how it looks on my machine:
>
> I can reproduce the issue Michael and Rados??aw are having:
>
> reima@marvin:~$ uname -a
> Linux marvin 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686
> GNU/Linux
> reima@marvin:~$ ruby -v
> ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
> reima@marvin:~$ irb
> >> require 'localmemcache'
> => true
> >> $lm = LocalMemCache.new :namespace => "viewcounters"
> => #<LocalMemCache:0xb7b69bdc>
> >> $lm.keys
> => []
> >> $lm[:foo]
> => nil
> >> $lm[:foo] = 1
> => 1
> >> $lm[:foo]
> => nil
> >> $lm.keys
> => ["foo"]
> >> puts `du -h /var/tmp/localmemcache/viewcounters.lmc`
> 16K /var/tmp/localmemcache/viewcounters.lmc
> => nil
> >>
>
> And on a sidenote:
>
> reima@marvin:~$ irb
> >> require 'localmemcache'
> => true
> >> $lm = LocalMemCache.new :namespace => "viewcounters"
> => #<LocalMemCache:0xb79eac20>
> >> $lm.close
> => nil
> >> $lm.keys
> (irb):4: [BUG] Segmentation fault
> ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
>
> Aborted
> reima@marvin:~$
>
> -Matthias

comp.lang.ruby

[ANN] Localmemcache-0.2.0: The beauty of memcached. For local data. Blazingly fast.

Sven C. Koehler

ara.t.howard

Bill Kelly

Sven C. Koehler

Michael Fellinger

Radoslaw Bulat

Sven C. Koehler

Sven C. Koehler

Radoslaw Bulat

Matthias Reitinger

Sven C. Koehler

x Login to ForumsZone