[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Re: Multi-dimensioned sparse array ?

Austin Ziegler

11/19/2003 7:03:00 AM

On Wed, 19 Nov 2003 14:21:19 +0900, Charles Hixson wrote:
> Does anyone have an implementation of a multi-dimensioned sparse
> array?

What about using hashes?

msa = Hash.new { |h, k| h[k] = Hash.new }

If you just use Numeric keys, there's little difference for your
needs. You can subclass Hash to make it act a bit more like an Array
under certain circumstances (e.g., #each would return values in
key-order instead of a key-value pair; you could also do an ordered
hash per [ruby-talk:20551] to potentially save sort times).

For multi-dimensional auto-vivifying, you could do:

hash_maker = proc { |h, k| h[k] = Hash.new(&hash_maker) }
msa = Hash.new(&hash_maker)

http://www.rubygarden.org/ruby?Ha...

-austin
--
austin ziegler * austin@halostatue.ca * Toronto, ON, Canada
software designer * pragmatic programmer * 2003.11.19
* 01.34.12


7 Answers

Robert Klemme

11/19/2003 5:39:00 PM

0


"Austin Ziegler" <austin@halostatue.ca> schrieb im Newsbeitrag
news:200311192229.932403@PADD...
> On Wed, 19 Nov 2003 14:21:19 +0900, Charles Hixson wrote:
> > Does anyone have an implementation of a multi-dimensioned sparse
> > array?
>
> What about using hashes?
>
> msa = Hash.new { |h, k| h[k] = Hash.new }
>
> If you just use Numeric keys, there's little difference for your
> needs. You can subclass Hash to make it act a bit more like an Array
> under certain circumstances (e.g., #each would return values in
> key-order instead of a key-value pair; you could also do an ordered
> hash per [ruby-talk:20551] to potentially save sort times).
>
> For multi-dimensional auto-vivifying, you could do:
>
> hash_maker = proc { |h, k| h[k] = Hash.new(&hash_maker) }
> msa = Hash.new(&hash_maker)

Another option: use a single Hash, *no* hash nesting and arrays as keys:

irb(main):012:0> multDim = Hash.new(0)
=> {}
irb(main):013:0> multDim[[1,12]]=4
=> 4
irb(main):014:0> multDim[[1,12]]=6
=> 6
irb(main):015:0> multDim[[3,12]]=5
=> 5
irb(main):016:0> multDim[[1,3]]=34
=> 34
irb(main):017:0> multDim[[1,12]]
=> 6
irb(main):018:0> multDim[[88,66]]
=> 0
irb(main):019:0>

Which of the two has better performance depends on the nature of algorithms
you want to do on the matrix.

If you subclass Hash and add some methods you can add dimension checks for
the keys, some fancy iteration stuff, create vectors on the fly and whatever
methods you need for the matrix...

Regards

robert

> http://www.rubygarden.org/ruby?Ha...
>
> -austin
> --
> austin ziegler * austin@halostatue.ca * Toronto, ON, Canada
> software designer * pragmatic programmer * 2003.11.19
> * 01.34.12
>
>

Charles Hixson

11/19/2003 6:43:00 PM

0

Austin Ziegler wrote:

>On Wed, 19 Nov 2003 14:21:19 +0900, Charles Hixson wrote:
>
>
>>Does anyone have an implementation of a multi-dimensioned sparse
>>array?
>>
>>
>
>What about using hashes?
>
> msa = Hash.new { |h, k| h[k] = Hash.new }
>
>If you just use Numeric keys, there's little difference for your
>needs. You can subclass Hash to make it act a bit more like an Array
>under certain circumstances (e.g., #each would return values in
>key-order instead of a key-value pair; you could also do an ordered
>hash per [ruby-talk:20551] to potentially save sort times).
>
>For multi-dimensional auto-vivifying, you could do:
>
> hash_maker = proc { |h, k| h[k] = Hash.new(&hash_maker) }
> msa = Hash.new(&hash_maker)
>
>http://www.rubygarden.org/ruby?Ha...
>
>-austin
>--
>austin ziegler * austin@halostatue.ca * Toronto, ON, Canada
>software designer * pragmatic programmer * 2003.11.19
> * 01.34.12
>
>
How would you index along other than the first dimension? But that's
one of the main requirements. (Currently I don't really need it to be
ordered, but I do need to be able to index it along more than one
dimension. One idea I had was to insert every indexed cell into a
sorted vector, and find the needed cell via binary search. But finding
the next cell along any particular index (except the last) would be an
entire new search. And since it's sparse this might need to be repeated
numerous times. Ugh! The list mesh looks faster, even though it
absorbe 2*n+1 units of memory for each cell used (two pointers for each
dimension.), and is itself quite slow.



Simon Strandgaard

11/19/2003 6:57:00 PM

0

>>On Wed, 19 Nov 2003 14:21:19 +0900, Charles Hixson wrote:
>>
>>
>>>Does anyone have an implementation of a multi-dimensioned sparse
>>>array?
>>>
[snip]
> numerous times. Ugh! The list mesh looks faster, even though it
> absorbe 2*n+1 units of memory for each cell used (two pointers for each
> dimension.), and is itself quite slow.

I am curious to how many dimension you need? 3, 7, 50 ?

Which kind of algorithm are you implementing?

--
Simon Strandgaard

Charles Hixson

11/19/2003 9:00:00 PM

0

Simon Strandgaard wrote:

>>>On Wed, 19 Nov 2003 14:21:19 +0900, Charles Hixson wrote:
>>>
>>>
>>>
>>>
>>>>Does anyone have an implementation of a multi-dimensioned sparse
>>>>array?
>>>>
>>>>
>>>>
>[snip]
>
>
>>numerous times. Ugh! The list mesh looks faster, even though it
>>absorbe 2*n+1 units of memory for each cell used (two pointers for each
>>dimension.), and is itself quite slow.
>>
>>
>
>I am curious to how many dimension you need? 3, 7, 50 ?
>
>Which kind of algorithm are you implementing?
>
>--
>Simon Strandgaard
>
It's actually basically a lookup algorithm at this point, but I'd rather
not be too specialized. If you want a simple visual idea, imagine that
you're representing the pieces on a chess board. One representation
could have columns, rows, and move-number, such that if you knew the
column, row, and move number you could quickly determine which (if any)
piece was on that square. I'm actually considering many more than eight
rows & columns, and more than 200 moves, and perhaps there'll be a third
(unknown as yet) dimension. Like, perhaps, in a game of Angband. I
realize that none of these examples are done quite this way, for rather
obvious reasons. One would clearly need to pack the representation
tightly for storage...but that's no big problem, as when it's being
stored one doesn't need to move between rows & columns. The packing and
unpacking operations can afford to be relatively slow.

The end goal of the structure is to be included in an AI program which I
haven't yet designed, much less built. But I'm looking for useful
pieces that I can build before starting. E.g. I know that I'll need a
B+Tree. And I suspect that I'll need the data structure to have
multiple indexes, so a simple B+Tree approach won't work. (This is
nearly unrelated to the multi-dimensioned sparse array.) Anyway, I
don't know how many dimensions I'll need, but probably no more than four
large ones (where large means to big to be a reasonable enumeration in C).

If this approach proves too difficult, I'll probably settle for using a
packed representation, with reasoning being done using an unpacked fixed
size array (perhaps 500 X 500 x 10 or 100 X 100 X 100 X 10) that I might
call the foeva, but as you can see it would quickly get too large to be
useful. Another approach would be to do all of the detail work in, say,
D (Digital Mars D), a language that is in many ways similar to Ruby, but
is an expansion from C that differs from either C++ or Objective C.
(Objective C is another possibility, but I've never learned it even as
well as I've learned D -- which is still being written.)


Simon Strandgaard

11/19/2003 10:03:00 PM

0

On Thu, 20 Nov 2003 06:00:03 +0900, Charles Hixson wrote:

> Simon Strandgaard wrote:
>
>>>>On Wed, 19 Nov 2003 14:21:19 +0900, Charles Hixson wrote:
>>>>
>>>>>Does anyone have an implementation of a multi-dimensioned sparse
>>>>>array?
>>>>>
>>>
>>
>>I am curious to how many dimension you need? 3, 7, 50 ?
>>
>>Which kind of algorithm are you implementing?
[snip]
> column, row, and move number you could quickly determine which (if any)
> piece was on that square. I'm actually considering many more than eight
> rows & columns, and more than 200 moves, and perhaps there'll be a third
> (unknown as yet) dimension. Like, perhaps, in a game of Angband. I


How about just using a one-dimentional sparse array, and this many2one
dimension converter ?


server> ruby a.rb
0
4
16
63
server> expand -t2 a.rb
class SparseMatrix
def initialize(*size)
@size = size
@data = nil
end
attr_accessor :data
def calc_index(*where) # many2one dimension conversion
index = 0
where.each_with_index{|val, i|
index *= @size[i]
index += val
}
index
end
def [](*where)
@data[calc_index(*where)]
end
end

m = SparseMatrix.new(4, 4, 4)
m.data = (0..63).to_a # TODO: use sparse matrix array instead
p m[0, 0, 0] # 0
p m[0, 1, 0] # 4
p m[1, 0, 0] # 16
p m[3, 3, 3] # 63
server>

--
Simon Strandgaard

Charles Hixson

11/20/2003 12:34:00 AM

0

Simon Strandgaard wrote:

>On Thu, 20 Nov 2003 06:00:03 +0900, Charles Hixson wrote:
>
>
>
>>Simon Strandgaard wrote:
>>
>>
>>
>>>>>On Wed, 19 Nov 2003 14:21:19 +0900, Charles Hixson wrote:
>>>>>
>>>>>
>>>>>
>>>>>>Does anyone have an implementation of a multi-dimensioned sparse
>>>>>>array?
>>>>>>
>>>>>>
>>>>>>
>>>I am curious to how many dimension you need? 3, 7, 50 ?
>>>
>>>Which kind of algorithm are you implementing?
>>>
>>>
>[snip]
>
>
>>column, row, and move number you could quickly determine which (if any)
>>piece was on that square. I'm actually considering many more than eight
>>rows & columns, and more than 200 moves, and perhaps there'll be a third
>>(unknown as yet) dimension. Like, perhaps, in a game of Angband. I
>>
>>
>
>
>How about just using a one-dimentional sparse array, and this many2one
>dimension converter ?
>
>
>server> ruby a.rb
>0
>4
>16
>63
>server> expand -t2 a.rb
>class SparseMatrix
> def initialize(*size)
> @size = size
> @data = nil
> end
> attr_accessor :data
> def calc_index(*where) # many2one dimension conversion
> index = 0
> where.each_with_index{|val, i|
> index *= @size[i]
> index += val
> }
> index
> end
> def [](*where)
> @data[calc_index(*where)]
> end
>end
>
>m = SparseMatrix.new(4, 4, 4)
>m.data = (0..63).to_a # TODO: use sparse matrix array instead
>p m[0, 0, 0] # 0
>p m[0, 1, 0] # 4
>p m[1, 0, 0] # 16
>p m[3, 3, 3] # 63
>server>
>
>--
>Simon Strandgaard
>
That *is* an interesting basic approach. Of course data would need to
be a hash table, but it certainly gets around the worries about the
multiplicity of hash tables using excessive memory. I'd just need to
set the size large enough (say around 191to allow for 4 dimensions being
handled with a single word index...but that's too small for many uses,
so I'd often need to use a Bignum rather than a Fixnum). Unfortunately,
for my main use I expect I'll need an index that goes upto "around"
5,000,000, so Bignums it will need to be. Still, a calculation is much
faster than the indirect lookups that were being talked about, and the
actual hash would only need to store the entries that were actually
generated....MUCH nicer! And that hash clearly won't generate instances
just because you try to read them, another big plus. AND it's easier to
understand. You're pretending to have the full array available, when
you look it's there, if you store something, it's remembered. Nice!
The only advantage that the mesh had was that you didn't need to check
the empty cells to move up and down in a column (or row), and that was
counterbalenced by not being able to find the row unless you started at
a known position, and needing linear access.

So good in fact that all I need to do is decide between a hash and an
AVL tree. I probably won't need sorted access, so that's a clear vote
in favor of the hash (and it won't cost much to add later if I decide I
need it, so that's another vote for the hash). The one real problem
is that resizing the array would be quite expensive, so I'll need to
make sure that it's large enough when the addresses are first
calculated. Again Bignum indexes. (But I *do* wish that AVL trees were
built-ins with a clear literal representation .. and all the hash
operators implemented. It's not the data structure that's
problematical, it's the literal representation...something analogous to
the way hash literals are represented.)



Robert Klemme

11/20/2003 8:59:00 AM

0


"Charles Hixson" <charleshixsn@earthlink.net> schrieb im Newsbeitrag
news:3FBBD914.7050602@earthlink.net...
> Simon Strandgaard wrote:
>
> >>>On Wed, 19 Nov 2003 14:21:19 +0900, Charles Hixson wrote:
> >>>
> >>>
> >>>
> >>>
> >>>>Does anyone have an implementation of a multi-dimensioned sparse
> >>>>array?
> >>>>
> >>>>
> >>>>
> >[snip]
> >
> >
> >>numerous times. Ugh! The list mesh looks faster, even though it
> >>absorbe 2*n+1 units of memory for each cell used (two pointers for
each
> >>dimension.), and is itself quite slow.
> >>
> >>
> >
> >I am curious to how many dimension you need? 3, 7, 50 ?
> >
> >Which kind of algorithm are you implementing?
> >
> >--
> >Simon Strandgaard
> >
> It's actually basically a lookup algorithm at this point,

Then I'd really consider the "Hash with Arrays as keys" approach (see my
other post). Lookups should be quite fast.

Regards

robert