Asp Forum - Re: Is iterating in lock-step possible?

Roshan James

3/11/2005 6:57:00 AM

Thanks for all the mails. Like William called out, this did not solve
the underlying problem.

<Repeat>
In my post, I chose the array.each only as an example. Its not arrays
for which I want to do this with - I want to solve the problem for
iterator calls. If zip() internally creates a list of values from both
iterations, then it does not help me. I want to be able to do the actual
computation of the iterator calls in lock-step.
</Repeat>

I was trying to write code where I can have two or more computations
which generate values that I need to compare - with the ruby syntax
tying iterator usage to the code blocks by syntax, there is no way I can
do this. This is especially applicable when the iterators potentially
return an infinite stream of values. (Except for using contiuations...
Which is not really so much of a solution, because I might as well not
have used an iterator in a first place if I could create good
coroutines)

Yes I understand that I take a hit (wrt perf by choosing ruby), but
that's a BAD argument to favour wrapping state first by a iterators and
then once over by a continuation.

On a related note, I had asked this question about C# and it looks like
C# can do both of these cleanly -
foreach(<type> t in foo())
{
foreach(<type> t in bar())
{
//nested
}
}

And -

IEnumerator<int> ie1 = a.foo().GetEnumerator();
IEnumerator<int> ie2 = a.bar().GetEnumerator();

while (ie1.MoveNext() && ie2.MoveNext())
{
//lockstep
}

Since Ruby is open to (I believe) a major language redesign these days,
this might be a good feature to consider adding.

Roshan

7 Answers

Robert Klemme

3/11/2005 9:22:00 AM

"Roshan James" <roshanj@microsoft.com> schrieb im Newsbeitrag
news:E8AA887D9A078F45B870125A37ED234701BC99A0@APS-MSG-02.southpacific.corp.microsoft.com...
> Thanks for all the mails. Like William called out, this did not solve
> the underlying problem.
>
> <Repeat>
> In my post, I chose the array.each only as an example. Its not arrays
> for which I want to do this with - I want to solve the problem for
> iterator calls. If zip() internally creates a list of values from both
> iterations, then it does not help me. I want to be able to do the actual
> computation of the iterator calls in lock-step.
> </Repeat>
>
> I was trying to write code where I can have two or more computations
> which generate values that I need to compare - with the ruby syntax
> tying iterator usage to the code blocks by syntax, there is no way I can
> do this. This is especially applicable when the iterators potentially
> return an infinite stream of values. (Except for using contiuations...
> Which is not really so much of a solution, because I might as well not
> have used an iterator in a first place if I could create good
> coroutines)
>
> Yes I understand that I take a hit (wrt perf by choosing ruby), but
> that's a BAD argument to favour wrapping state first by a iterators and
> then once over by a continuation.
>
> On a related note, I had asked this question about C# and it looks like
> C# can do both of these cleanly -
> foreach(<type> t in foo())
> {
> foreach(<type> t in bar())
> {
> //nested
> }
> }
>
> And -
>
> IEnumerator<int> ie1 = a.foo().GetEnumerator();
> IEnumerator<int> ie2 = a.bar().GetEnumerator();
>
> while (ie1.MoveNext() && ie2.MoveNext())
> {
> //lockstep
> }
>
>
> Since Ruby is open to (I believe) a major language redesign these days,
> this might be a good feature to consider adding.

I'm not sure about that one. The issue pops up once in a while but not
too often to justify a general change in iteration style. If it just was
an add on - maybe. But I guess the overhead is significant.

Since you are doing calculations, what prevents you from doing it the
other way round?

class SillyGenerator
def initialize() reset() end
def next?() @val < 1000 end
def reset() @val = 0 end
def move() @val += 2 end
def get() @val end
end

class Calculator
include Enumerable
def initialize(source) @source = source end
def each
@source.reset

while @source.next?
@source.move
yield @source.get
end

self
end
end

Or in one class

class BaseGenerator
def each
reset

while next?
move
yield get
end

self
end
end

class SillyGenerator < BaseGenerator
include Enumerable

def initialize() reset() end
def next?() @val < 1000 end
def reset() @val = 0 end
def move() @val += 2 end
def get() @val end
end

Now you can use it both ways:

>> SillyGenerator.new.inject(0){|max, e| max > e ? max : e}
=> 1000

iter1 = SillyGenerator.new
iter2 = SillyGenerator.new

while iter1.next? && iter2.next?
iter1.move
iter2.move
puts "failure" unless iter1.get == iter2.get
end

Kind regards

robert

William Morgan

3/11/2005 12:46:00 PM

Excerpts from Roshan James's mail of 11 Mar 2005 (EST):
> On a related note, I had asked this question about C# and it looks like
> C# can do both of these cleanly -
> foreach(<type> t in foo())
> {
> foreach(<type> t in bar())
> {
> //nested
> }
> }

I think you might be overlooking the obvious, which is that you can take
this approach in Ruby. You're writing the objects that do the
computations, right? Simply give them #next? and #next methods and use a
while loop. That's what you'd be doing in these other languages.

The prevalence of internal iterators in Ruby's API is simply a
reflection of the fact that they're so nice to use. It's not the only
way to iterate.

--
William <wmorgan-ruby-talk@masanjin.net>

Luke Graham

3/11/2005 1:42:00 PM

On Fri, 11 Mar 2005 15:57:08 +0900, Roshan James <roshanj@microsoft.com> wrote:

> I was trying to write code where I can have two or more computations
> which generate values that I need to compare - with the ruby syntax
> tying iterator usage to the code blocks by syntax, there is no way I can
> do this. This is especially applicable when the iterators potentially
> return an infinite stream of values. (Except for using contiuations...
> Which is not really so much of a solution, because I might as well not
> have used an iterator in a first place if I could create good
> coroutines)

Continuations are the right answer in this case.

module Enumerable
attr_accessor :co, :st, :res

def nextco
each { |e|
callcc { |cc|
@co = cc
@res = e
@st.call
}
}
@res = nil
@st.call
end
def next
callcc { |cc|
@st = cc
@co ? @co.call : nextco
}
return @res
end
end

a = [1,2,3]
b = [4,5,6,7,8]

while x = a.next and y = b.next
puts x,y
end

Note there are three possible behaviours around the @res = nil
line. One is to return nil after all items have been used up, another
is to keep returning the last item by removing that line, and finally
a loop can be wrapped around the each, to begin returning from
the first item again.

Its probably possible to do it in one function, but it makes more
sense to me in two.

> Yes I understand that I take a hit (wrt perf by choosing ruby), but
> that's a BAD argument to favour wrapping state first by a iterators and
> then once over by a continuation.

As long as its wrapped safely and nicely, what does it matter?
This way, you even get your choice of what to do when you
run out of items ;)

--
spooq

Luke Graham

3/11/2005 1:45:00 PM

Gah, always miss something when I post code...

Replace the first two lines of nextco with
def nextco
each_with_index { |e,i|

On Fri, 11 Mar 2005 23:41:46 +1000, Luke Graham <spoooq@gmail.com> wrote:
> On Fri, 11 Mar 2005 15:57:08 +0900, Roshan James <roshanj@microsoft.com> wrote:
>
> > I was trying to write code where I can have two or more computations
> > which generate values that I need to compare - with the ruby syntax
> > tying iterator usage to the code blocks by syntax, there is no way I can
> > do this. This is especially applicable when the iterators potentially
> > return an infinite stream of values. (Except for using contiuations...
> > Which is not really so much of a solution, because I might as well not
> > have used an iterator in a first place if I could create good
> > coroutines)
>
> Continuations are the right answer in this case.
>
> module Enumerable
> attr_accessor :co, :st, :res
>
> def nextco
> each { |e|
> callcc { |cc|
> @co = cc
> @res = e
> @st.call
> }
> }
> @res = nil
> @st.call
> end
> def next
> callcc { |cc|
> @st = cc
> @co ? @co.call : nextco
> }
> return @res
> end
> end
>
> a = [1,2,3]
> b = [4,5,6,7,8]
>
> while x = a.next and y = b.next
> puts x,y
> end
>
> Note there are three possible behaviours around the @res = nil
> line. One is to return nil after all items have been used up, another
> is to keep returning the last item by removing that line, and finally
> a loop can be wrapped around the each, to begin returning from
> the first item again.
>
> Its probably possible to do it in one function, but it makes more
> sense to me in two.
>
> > Yes I understand that I take a hit (wrt perf by choosing ruby), but
> > that's a BAD argument to favour wrapping state first by a iterators and
> > then once over by a continuation.
>
> As long as its wrapped safely and nicely, what does it matter?
> This way, you even get your choice of what to do when you
> run out of items ;)
>
> --
> spooq
>

--
spooq

Luke Graham

3/11/2005 1:59:00 PM

Just found this, it seems to fit the bill nicely and its in the std
lib of all places :P

http://www.ruby-doc.org/stdlib/libdoc/generator/rdoc/classes/Gene...

On Fri, 11 Mar 2005 23:44:39 +1000, Luke Graham <spoooq@gmail.com> wrote:
> Gah, always miss something when I post code...
>
> Replace the first two lines of nextco with
> def nextco
> each_with_index { |e,i|
>
>
> On Fri, 11 Mar 2005 23:41:46 +1000, Luke Graham <spoooq@gmail.com> wrote:
> > On Fri, 11 Mar 2005 15:57:08 +0900, Roshan James <roshanj@microsoft.com> wrote:
> >
> > > I was trying to write code where I can have two or more computations
> > > which generate values that I need to compare - with the ruby syntax
> > > tying iterator usage to the code blocks by syntax, there is no way I can
> > > do this. This is especially applicable when the iterators potentially
> > > return an infinite stream of values. (Except for using contiuations...
> > > Which is not really so much of a solution, because I might as well not
> > > have used an iterator in a first place if I could create good
> > > coroutines)
> >
> > Continuations are the right answer in this case.
> >
> > module Enumerable
> > attr_accessor :co, :st, :res
> >
> > def nextco
> > each { |e|
> > callcc { |cc|
> > @co = cc
> > @res = e
> > @st.call
> > }
> > }
> > @res = nil
> > @st.call
> > end
> > def next
> > callcc { |cc|
> > @st = cc
> > @co ? @co.call : nextco
> > }
> > return @res
> > end
> > end
> >
> > a = [1,2,3]
> > b = [4,5,6,7,8]
> >
> > while x = a.next and y = b.next
> > puts x,y
> > end
> >
> > Note there are three possible behaviours around the @res = nil
> > line. One is to return nil after all items have been used up, another
> > is to keep returning the last item by removing that line, and finally
> > a loop can be wrapped around the each, to begin returning from
> > the first item again.
> >
> > Its probably possible to do it in one function, but it makes more
> > sense to me in two.
> >
> > > Yes I understand that I take a hit (wrt perf by choosing ruby), but
> > > that's a BAD argument to favour wrapping state first by a iterators and
> > > then once over by a continuation.
> >
> > As long as its wrapped safely and nicely, what does it matter?
> > This way, you even get your choice of what to do when you
> > run out of items ;)
> >
> > --
> > spooq
> >
>
> --
> spooq
>

--
spooq

Luke Graham

3/11/2005 2:14:00 PM

More or less compliant with the generator.rb api, which I like.
Some proof that the thing works, too.

module Enumerable
attr_accessor :co, :st, :res, :sto

def nextco
each_with_index { |e,i|
callcc { |cc|
@co = cc
@res = e
@st.call
}
}
@res = nil
@st.call
end
def next
(t = @sto; @sto = nil; return t) if @sto
callcc { |cc|
@st = cc
@co ? @co.call : nextco
}
return @res
end
def next?
@sto = self.next if !@sto
@sto
end
end

a = [1,2,4]
b = [3,5,'lots']

puts a.next
b.next?
b.next?
b.next?

while a.next? and b.next?
puts a.next,b.next
end

On Fri, 11 Mar 2005 23:58:30 +1000, Luke Graham <spoooq@gmail.com> wrote:
> Just found this, it seems to fit the bill nicely and its in the std
> lib of all places :P
>
> http://www.ruby-doc.org/stdlib/libdoc/generator/rdoc/classes/Gene...
>
> On Fri, 11 Mar 2005 23:44:39 +1000, Luke Graham <spoooq@gmail.com> wrote:
> > Gah, always miss something when I post code...
> >
> > Replace the first two lines of nextco with
> > def nextco
> > each_with_index { |e,i|
> >
> >
> > On Fri, 11 Mar 2005 23:41:46 +1000, Luke Graham <spoooq@gmail.com> wrote:
> > > On Fri, 11 Mar 2005 15:57:08 +0900, Roshan James <roshanj@microsoft.com> wrote:
> > >
> > > > I was trying to write code where I can have two or more computations
> > > > which generate values that I need to compare - with the ruby syntax
> > > > tying iterator usage to the code blocks by syntax, there is no way I can
> > > > do this. This is especially applicable when the iterators potentially
> > > > return an infinite stream of values. (Except for using contiuations...
> > > > Which is not really so much of a solution, because I might as well not
> > > > have used an iterator in a first place if I could create good
> > > > coroutines)
> > >
> > > Continuations are the right answer in this case.
> > >
> > > module Enumerable
> > > attr_accessor :co, :st, :res
> > >
> > > def nextco
> > > each { |e|
> > > callcc { |cc|
> > > @co = cc
> > > @res = e
> > > @st.call
> > > }
> > > }
> > > @res = nil
> > > @st.call
> > > end
> > > def next
> > > callcc { |cc|
> > > @st = cc
> > > @co ? @co.call : nextco
> > > }
> > > return @res
> > > end
> > > end
> > >
> > > a = [1,2,3]
> > > b = [4,5,6,7,8]
> > >
> > > while x = a.next and y = b.next
> > > puts x,y
> > > end
> > >
> > > Note there are three possible behaviours around the @res = nil
> > > line. One is to return nil after all items have been used up, another
> > > is to keep returning the last item by removing that line, and finally
> > > a loop can be wrapped around the each, to begin returning from
> > > the first item again.
> > >
> > > Its probably possible to do it in one function, but it makes more
> > > sense to me in two.
> > >
> > > > Yes I understand that I take a hit (wrt perf by choosing ruby), but
> > > > that's a BAD argument to favour wrapping state first by a iterators and
> > > > then once over by a continuation.
> > >
> > > As long as its wrapped safely and nicely, what does it matter?
> > > This way, you even get your choice of what to do when you
> > > run out of items ;)
> > >
> > > --
> > > spooq
> > >
> >
> > --
> > spooq
> >
>
> --
> spooq
>

--
spooq

Cs. Henk

3/11/2005 5:48:00 PM

On Fri, Mar 11, 2005 at 11:13:54PM +0900, Luke Graham wrote:
> More or less compliant with the generator.rb api, which I like.
> Some proof that the thing works, too.
>
> module Enumerable
> attr_accessor :co, :st, :res, :sto
>

(snip)

> end

I also wrote some lightweight generator, before I knew that there is one
in the std lib.

Now I see that it's about the same speed as your Enumerable extension,
ie., they are cca. 1.5 times faster than the official one.

This is enough reason to show it... it converts an iterator method to a
generator. Eg.:

require 'iter2gener'

a = (0...3000).to_a
g = I2G.new a.method(:each)
while e=g.next; p e; end

You have some further options, eg, you can set what to do when iterator
is exhausted.

Csaba

comp.lang.ruby

Re: Is iterating in lock-step possible?

Roshan James

Robert Klemme

William Morgan

Luke Graham

Luke Graham

Luke Graham

Luke Graham

Cs. Henk

x Login to ForumsZone