Asp Forum - Re: TupleSpace performance (TupleBag really

Eric Hodel

2/8/2007 1:24:00 AM

On Feb 7, 2007, at 09:21, Mark Alexander Friedgan wrote:

> We've been struggling with this problem for months. We use
> TupleSpace to
> implement a distributed processing framework and periodically if
> the number
> of objects
> in the TupleBag gets large (not ridiculous but 50,000 or so) the
> TupleSpace
> begins to take 100pct cpu and is effectively neutered and must be
> restarted.
> We've been trying to come up
> with a better implementation of TupleBag but have not had much luck
> so far.
> Has anyone else done this?

I've heard a few people talk of it (mainly, TupleSpace with a
persistence (database)), but to my knowledge nobody's done it.

6 Answers

Ara.T.Howard

2/10/2007 12:05:00 AM

Bob Hutchison

2/10/2007 1:49:00 AM

On 9-Feb-07, at 7:04 PM, ara.t.howard@noaa.gov wrote:

> On Thu, 8 Feb 2007, Eric Hodel wrote:
>
>>
>> I've heard a few people talk of it (mainly, TupleSpace with a
>> persistence (database)), but to my knowledge nobody's done it.
>>
>
> i have. if people like it, i'll release next week.

What version/implementation of sqlite are you using?

Cheers,
Bob

----
Bob Hutchison -- blogs at <http://www.rec...
hutch/>
Recursive Design Inc. -- <http://www.rec...>
Raconteur -- <http://www.raconteur...
xampl for Ruby -- <http://rubyforge.org/projects/...

Ara.T.Howard

2/10/2007 2:11:00 AM

Mauricio Fernández

2/10/2007 9:13:00 AM

On Sat, Feb 10, 2007 at 09:04:51AM +0900, ara.t.howard@noaa.gov wrote:
> On Thu, 8 Feb 2007, Eric Hodel wrote:
> >
> >I've heard a few people talk of it (mainly, TupleSpace with a persistence
> >(database)), but to my knowledge nobody's done it.
> >
> i have. if people like it, i'll release next week.
>
> 1) it's a hack
> 2) it's persistent
[...]
> 3) it's pretty fast (and uses little cpu)
>
> jib:~ > cat b.rb
> require 'sqlitespace'
>
> rss = Rinda::SQLiteSpace.new
>
> rss.transaction{ 42424.times{ write [Time.now, rand] } }
>
>
> jib:~ > time ruby b.rb
>
> real 0m19.063s
> user 0m18.110s
> sys 0m0.560s

It's pretty fast considering it's persistent, but it is of course slower than
the std. TupleSpace...

$ cat tuplespace.rb
#!/usr/bin/env ruby

require 'rinda/tuplespace'

ts = Rinda::TupleSpace.new

require 'benchmark'
p Benchmark.measure {
42424.times{ ts.write [Time.now, rand] }
}

#require 'rubygems'
#gem "sqlite-ruby"
require 'sqlitespace'
$VERBOSE = false
puts "Using SQLite #{SQLite::Version::STRING}"
ts2 = Rinda::SQLiteSpace.new

p Benchmark.measure {
ts2.transaction{ 42424.times{ write [Time.now, rand] } }
}

$ time ruby tuplespace.rb
#<Benchmark::Tms:0xa795b878 @total=4.62, @cutime=0.0, @label="", @stime=0.24, @real=4.64543700218201, @utime=4.38, @cstime=0.0>
/home/batsman/usr//lib/ruby/site_ruby/1.8/sqlite/database.rb:243: warning: `*' interpreted as argument prefix
/home/batsman/usr//lib/ruby/site_ruby/1.8/sqlite/statement.rb:103: warning: `*' interpreted as argument prefix
(eval):2: warning: `*' interpreted as argument prefix
(eval):2: warning: `*' interpreted as argument prefix
(eval):2: warning: `*' interpreted as argument prefix
(eval):2: warning: `*' interpreted as argument prefix
(eval):2: warning: `*' interpreted as argument prefix
(eval):2: warning: `*' interpreted as argument prefix
(eval):1: warning: method redefined; discarding old delete
Using SQLite 2.2.2
#<Benchmark::Tms:0xa7717d8c @total=74.8, @cutime=0.0, @label="", @stime=1.79, @real=76.8895101547241, @utime=73.01, @cstime=0.0>

real 1m21.836s
user 1m17.625s
sys 0m2.060s

(it seems slower w/o .transaction)

... and it won't scale any better, so it wouldn't solve the OP's problem:

> def find_all(template, limit=nil, &block)
> template= Rinda::TemplateEntry.new(Array.new(template)) if Numeric
> === template
> size = template.size
> all = []
> @db.execute "select blob from bag where size=#{ size } #{ limit ?
> ('limit=%d' % limit) : '' }" do |tuple|
> blob = tuple['blob']
> t = decode blob
> if t.alive? && template.match(t)
> block ? block[t] : (all << t)
> end
> end
> block ? nil : all
> end
[...]
> def find_all_template(tuple, limit=nil, &block)
> size = tuple.size
> all = []
> @db.execute "select blob from bag where size=#{ size } #{ limit ?
> ('limit=%d' % limit) : '' }" do |tuple|
> blob = tuple['blob']
> t = decode blob
> if t.alive? && t.match(tuple)
> block ? block[t] : (all << t)
> end
> end
> block ? nil : all
> end

PS: some warnings removed...

--- sqlitespace.rb.orig 2007-02-10 09:33:20.000000000 +0100
+++ sqlitespace.rb 2007-02-10 10:09:50.000000000 +0100
@@ -43,7 +43,7 @@
@read_waiter.transaction{
@take_waiter.transaction{
@notify_waiter.transaction{
- instance_eval &b
+ instance_eval(&b)
}
}
}
@@ -79,7 +79,7 @@
def setup
begin
@db.execute(SCHEMA){}
- rescue ::SQLite::SQLException => e
+ rescue (SQLite::SQLException rescue ::SQLite::Exceptions::SQLException) # tested with sqlite-ruby 2.2.3
42
end
end
@@ -88,7 +88,7 @@
# for older rubys
#
begin
- Base64
+ x = Base64
def encode64 blob
Base64.encode64 blob
end
@@ -122,7 +122,7 @@
end

def execute *a, &b
- @db.execute *a, &b
+ @db.execute(*a, &b)
end

def transaction *a, &b
@@ -213,7 +213,7 @@
end

def transaction *a, &b
- @db.transaction *a, &b
+ @db.transaction(*a, &b)
end

%w( push delete find_all find find_all_template delete ).each do |m|

--
Mauricio Fernandez - http://eige... - singular Ruby
** Latest postings **
What's new in Ruby 1.9, Feb. 07 update
http://eige.../hiki.rb?Changes-in-Ruby-1.9-update-6
Firebrigade: automated, sandboxed testing of RubyGems packages by other devels
http://eige.../hiki.rb?firebrigade-launched

Bob Hutchison

2/10/2007 3:02:00 PM

On 9-Feb-07, at 10:01 PM, ara.t.howard@noaa.gov wrote:

> This message is in MIME format. The first part should be
> readable text,
> while the remaining parts are likely unreadable without MIME-
> aware tools.
> On Sat, 10 Feb 2007, Bob Hutchison wrote:
>
>>
>> On 9-Feb-07, at 7:04 PM, ara.t.howard@noaa.gov wrote:
>>
>>> On Thu, 8 Feb 2007, Eric Hodel wrote:
>>>> I've heard a few people talk of it (mainly, TupleSpace with a
>>>> persistence (database)), but to my knowledge nobody's done it.
>>> i have. if people like it, i'll release next week.
>>
>> What version/implementation of sqlite are you using?
>
> give this one a whirl...

Thanks.

Okay I admit that I never quite worked out what was going on with the
sqlite packages in ruby. There are two it seems in the same project:
sqlite and sqlite3. And I think there is another project out there
someplace.

Trying to install sqlite as a gem installs version 2.0.1 but the most
recent version, if I understand rubyforge correctly, is 2.2.3; and
2.0.1 is marked a beta. Even though gem install tries to install
2.0.1 there is a gem on rubyforge for 2.2.3.

Worse, 2.0.1 won't compile on my machine. I havn't looked into it yet
but it reeks of missing libraries.

Sqlite3 does appear to install. I know I've had it working before so
this doesn't surprise me too much (this is also the reason I never
worked out what the different projects are).

Oh well. Something to do on a Saturday morning. And the delayed
gratification in seeing your 'hack' isn't all it's cracked up to be,
I think I prefer instant gratification -- but gratification there
will be :-)

Cheers,
Bob

>
> -a
> --
> we can deny everything, except that we have the possibility of
> being better.
> simply reflect on that.
> - the dalai lama
> <sqlitespace.rb>

----
Bob Hutchison -- blogs at <http://www.rec...
hutch/>
Recursive Design Inc. -- <http://www.rec...>
Raconteur -- <http://www.raconteur...
xampl for Ruby -- <http://rubyforge.org/projects/...

Ara.T.Howard

2/10/2007 5:53:00 PM

comp.lang.ruby

Re: TupleSpace performance (TupleBag really

Eric Hodel

Ara.T.Howard

Bob Hutchison

Ara.T.Howard

Mauricio Fernández

Bob Hutchison

Ara.T.Howard

x Login to ForumsZone