Asp Forum - Re: array comparison

Brian Candler

10/30/2008 9:08:00 AM

Chad Perrin wrote:
> I tend to guess that loading each list
> into an array and running a direct comparison of them:
>
> array_1 = [millions of things]
> array_2 = [millions of things]
> array_3 = array1 & array2
>
> . . . would fill up RAM in a hurry and drag system performance on a
> typical desktop computer to a standstill. What sort of approach would
> the expert Ruby hackers suggest for achieving much the same ends without
> taking all week and risking a stack overflow?

If these millions of things are being read from disk, then sort them
first (*). Then you can do a sort-merge to check which are in one but
not the other.

Roughly: Read the first item from both and compare them. If they're
equal, munch them both. If the item from A is smaller than the item in
B, then munch the item from A (since it exists in A but not B), and vice
versa. An efficient implementation might read a few thousands items from
both A and B into local buffers first.

The Sedgewick "Algorithms" book has chapters on external sorting and
searching.

HTH,

Brian.

(*) Of course, the problem then becomes one of sorting millions of
things, but this is a well-solved problem. The unix "sort" command-line
tool will happily sort files which are much larger than available RAM,
by dividing into smaller chunks and sort-merging them. Just take care to
set LC_ALL properly; otherwise sort behaves in a very strange way.
LC_ALL=C seems to work best for me.

Once the inputs are sorted, an '&' or '|' operation implemented as above
will generate an output which is already sorted.

The unix 'join' command can do these operations for you, too.
--
Posted via http://www.ruby-....

31 Answers

Jeff

11/2/2010 9:55:00 PM

On Nov 2, 4:50 pm, scouser <denise.theophi...@yahoo.co.uk> wrote:
> On Nov 2, 9:05 pm, Nil <redno...@REMOVETHIScomcast.net> wrote:
>
>
>
>
>
>
>
>
>
> > On 02 Nov 2010, poisoned rose <pro...@wipingbottoms.com> wrote in
> > rec.music.beatles:
>
> > > I just can't understand how they did 102 takes and ended up with
> > > nothing better than THAT gutless trudge. I can easily see why the
> > > song was tossed, at least in that form.
>
> > > Maybe it should have been pushed through the same ballsy,
> > > fuzzy-rock filter which yielded the single version of
> > > "Revolution." Because the song just lays down and dies, as is. But
> > > if the group really did 102 takes, I guess they futilely tried to
> > > make it work in a number of ways.
>
> > I think that recording stinks pretty badly, too. It's not a bad song,
> > and George's later solo recording makes something attractive out of it,
> > but The Beatles never found a good groove for it and all takes of it
> > sound forced and uncomfortable to me. "Gutless trudge" describes it
> > pretty well. George's vocal is wan and pale as a ghost.
>
> it needed wanness and pale ghostliness...

It's terrible. The Beatles were trying to make it rock.

scouser

11/2/2010 9:56:00 PM

On Nov 2, 9:53 pm, "who?" <yourimageunre...@sbcglobal.net> wrote:
> On Nov 2, 4:49 pm, scouser <denise.theophi...@yahoo.co.uk> wrote:
>
>
>
>
>
> > On Nov 2, 8:56 pm, "who?" <yourimageunre...@sbcglobal.net> wrote:
>
> > > On Nov 2, 3:51 pm, scouser <denise.theophi...@yahoo.co.uk> wrote:
>
> > > > On Nov 2, 6:54 pm, poisoned rose <pro...@wipingbottoms.com> wrote:
>
> > > > > scouser <denise.theophi...@yahoo.co.uk> wrote:
> > > > > > a good
> > > > > > George song e.g Not Guilty which was destined for this album
>
> > > > > I just can't understand how they did 102 takes and ended up with nothing
> > > > > better than THAT gutless trudge. I can easily see why the song was
> > > > > tossed, at least in that form.
>
> > > > > Maybe it should have been pushed through the same ballsy, fuzzy-rock
> > > > > filter which yielded the single version of "Revolution." Because the
> > > > > song just lays down and dies, as is. But if the group really did 102
> > > > > takes, I guess they futilely tried to make it work in a number of ways.
>
> > > > "gutless trudge!" Ye gods strike the philistine ... presumably YOU
> > > > preferred every other tune which made the White Album to this?
>
> > > Let's talk about ATMP. It was better cause it had slide and Clapton
> > > on it. :-)- Hide quoted text -
>
> > > - Show quoted text -
>
> > i hate Clapton though...
>
> Why is that?- Hide quoted text -
>
> - Show quoted text -

i just think he is a pillock because of something i know.

scouser

11/2/2010 9:57:00 PM

On Nov 2, 9:29 pm, poisoned rose <pro...@wipingbottoms.com> wrote:
> "who?" <yourimageunre...@sbcglobal.net> wrote:
> > > > Let's talk about ATMP. It was better cause it had slide and Clapton
> > > > on it. :-)
>
> > > Still repeatedly trolling about issues you claim don't matter much to
> > > you, eh?
>
> > You're the one who looks down on the posters here, talking about
> > wiping bottoms. So....your post just makes me laugh.
>
> This post is a non sequitur.
>
> If you don't want to be picked on, stop seeking fights.

Be careful he will use his secateurs... they are very dangerous to
poisoned rosebushes. i know because i use them every spring...

Jeff

11/2/2010 9:59:00 PM

On Nov 2, 4:55 pm, scouser <denise.theophi...@yahoo.co.uk> wrote:
> On Nov 2, 9:53 pm, "who?" <yourimageunre...@sbcglobal.net> wrote:
>
>
>
>
>
>
>
>
>
> > On Nov 2, 4:49 pm, scouser <denise.theophi...@yahoo.co.uk> wrote:
>
> > > On Nov 2, 8:56 pm, "who?" <yourimageunre...@sbcglobal.net> wrote:
>
> > > > On Nov 2, 3:51 pm, scouser <denise.theophi...@yahoo.co.uk> wrote:
>
> > > > > On Nov 2, 6:54 pm, poisoned rose <pro...@wipingbottoms.com> wrote:
>
> > > > > > scouser <denise.theophi...@yahoo.co.uk> wrote:
> > > > > > > a good
> > > > > > > George song e.g Not Guilty which was destined for this album
>
> > > > > > I just can't understand how they did 102 takes and ended up with nothing
> > > > > > better than THAT gutless trudge. I can easily see why the song was
> > > > > > tossed, at least in that form.
>
> > > > > > Maybe it should have been pushed through the same ballsy, fuzzy-rock
> > > > > > filter which yielded the single version of "Revolution." Because the
> > > > > > song just lays down and dies, as is. But if the group really did 102
> > > > > > takes, I guess they futilely tried to make it work in a number of ways.
>
> > > > > "gutless trudge!" Ye gods strike the philistine ... presumably YOU
> > > > > preferred every other tune which made the White Album to this?
>
> > > > Let's talk about ATMP. It was better cause it had slide and Clapton
> > > > on it. :-)- Hide quoted text -
>
> > > > - Show quoted text -
>
> > > i hate Clapton though...
>
> > Why is that?- Hide quoted text -
>
> > - Show quoted text -
>
> i just think he is a pillock because of something i know.

My posting limit just expired, so I had to change to this
handle. Lots of posters posting today.

scouser

11/2/2010 10:01:00 PM

On Nov 2, 9:37 pm, "who?" <yourimageunre...@sbcglobal.net> wrote:
> On Nov 2, 4:29 pm, poisoned rose <pro...@wipingbottoms.com> wrote:
>
> > "who?" <yourimageunre...@sbcglobal.net> wrote:
> > > > > Let's talk about ATMP. It was better cause it had slide and Clapton
> > > > > on it. :-)
>
> > > > Still repeatedly trolling about issues you claim don't matter much to
> > > > you, eh?
>
> > > You're the one who looks down on the posters here, talking about
> > > wiping bottoms. So....your post just makes me laugh.
>
> > This post is a non sequitur.
>
> > If you don't want to be picked on, stop seeking fights.
>
> Stop taking things so seriously This day too shall pass.

Is the poisoned one a girl or boy? What do you think happened to him/
her to make him the way he/she is? Do you think there is a cure for
this or is it too late? After all we have an obligation to save poison
victims if we possibly can. If anyone can think of a remedy or an
antidote then maybe he/she will turn flesh coloured again... and we
can play Yellow Submarine and dance and sing with gay abandon...

scouser

11/2/2010 10:03:00 PM

On Nov 2, 9:28 pm, "who?" <yourimageunre...@sbcglobal.net> wrote:
> On Nov 2, 4:10 pm, poisoned rose <pro...@wipingbottoms.com> wrote:
>
> > "who?" <yourimageunre...@sbcglobal.net> wrote:
> > > Let's talk about ATMP. It was better cause it had slide and Clapton
> > > on it. :-)
>
> > Still repeatedly trolling about issues you claim don't matter much to
> > you, eh?
>
> You're the one who looks down on the posters here, talking about
> wiping bottoms. So....your post just makes me laugh.

Wiping bottoms? like in The Holy Mountain?

Jeff

11/2/2010 10:04:00 PM

On Nov 2, 5:02 pm, scouser <denise.theophi...@yahoo.co.uk> wrote:
> On Nov 2, 9:28 pm, "who?" <yourimageunre...@sbcglobal.net> wrote:
>
> > On Nov 2, 4:10 pm, poisoned rose <pro...@wipingbottoms.com> wrote:
>
> > > "who?" <yourimageunre...@sbcglobal.net> wrote:
> > > > Let's talk about ATMP. It was better cause it had slide and Clapton
> > > > on it. :-)
>
> > > Still repeatedly trolling about issues you claim don't matter much to
> > > you, eh?
>
> > You're the one who looks down on the posters here, talking about
> > wiping bottoms. So....your post just makes me laugh.
>
> Wiping bottoms? like in The Holy Mountain?

Click on his profile. It's part of his email address.

Jeff

11/2/2010 10:09:00 PM

On Nov 2, 5:01 pm, scouser <denise.theophi...@yahoo.co.uk> wrote:
> On Nov 2, 9:37 pm, "who?" <yourimageunre...@sbcglobal.net> wrote:
>
>
>
>
>
>
>
>
>
> > On Nov 2, 4:29 pm, poisoned rose <pro...@wipingbottoms.com> wrote:
>
> > > "who?" <yourimageunre...@sbcglobal.net> wrote:
> > > > > > Let's talk about ATMP. It was better cause it had slide and Clapton
> > > > > > on it. :-)
>
> > > > > Still repeatedly trolling about issues you claim don't matter much to
> > > > > you, eh?
>
> > > > You're the one who looks down on the posters here, talking about
> > > > wiping bottoms. So....your post just makes me laugh.
>
> > > This post is a non sequitur.
>
> > > If you don't want to be picked on, stop seeking fights.
>
> > Stop taking things so seriously This day too shall pass.
>
> Is the poisoned one a girl or boy?

Man.

What do you think happened to him/
> her to make him the way he/she is?

It's better we don't talk about him.

Nil

11/2/2010 10:12:00 PM

On 02 Nov 2010, scouser <denise.theophilus@yahoo.co.uk> wrote in
rec.music.beatles:

> See you said "Wild Honey Pie" ... not "Honey Pie"

Yeah. And...?

Nil

11/2/2010 10:13:00 PM

On 02 Nov 2010, scouser <denise.theophilus@yahoo.co.uk> wrote in
rec.music.beatles:

> it needed wanness and pale ghostliness...

Didn't work for me. The Whte Album would have been diminished had "Not
Guilty" appeared on it.

comp.lang.ruby

Re: array comparison

Brian Candler

Jeff

scouser

scouser

Jeff

scouser

scouser

Jeff

Jeff

Nil

Nil

x Login to ForumsZone