comp.lang.python

Arnaud Delobelle

3/5/2010 10:00:00 PM

mk <mrkafk@gmail.com> writes:

> Hello everyone,
>
> I re-wrote more "slowly" an example at the end of
> http://wordaligned.org/articles/essential-python-re...
>
>
> This example finds anagrams in the text file.
>
> ====
> from itertools import groupby, imap
> from operator import itemgetter
>
> from string import ascii_lowercase, ascii_uppercase, punctuation,
> maketrans, translate
>
> data = open(r"c:\temp\words.txt", "rt").read()
>
> trtable = maketrans(ascii_uppercase, ascii_lowercase)
>
> words = translate(data, trtable, deletions = punctuation)
>
> words = list(set(words.split()))
>
> sw = sorted(words, key=sorted)
>
> gb = groupby(sw, sorted)
>
> print map(list, imap(itemgetter(1), gb))
> ===
>
> words.txt:
> ===
> Word Aligned
> three
> space sensitive programming
> Feed Logo tins
> Essential Python post Reading List
> stop course there
> times isnt
> capes
> ===
>
> Now, when I execute above, it works:
>
> [['capes', 'space'], ['aligned'], ['reading'], ['essential'],
> ['programming'], ['course'], ['feed'], ['word'], ['there', 'three'],
> ['sensitive'], ['times'], ['logo'], ['python'], ['list'], ['isnt',
> tins'], ['stop', 'post']]
>
>
> However, when I change the last line to:
>
> print map(list, map(itemgetter(1), gb))
>
> It stops working:
>
> [[], [], [], [], [], [], [], [], [], [], [], [], [], [], [], ['post']]
>
> Why? I was under impression that the only difference between map and
> imap is that imap returns iterator allowing to produce a list, while
> map returns equivalent list?
>
> Regards,
> mk

From the itertools docs [1]:

The returned group is itself an iterator that shares the underlying
iterable with groupby(). Because the source is shared, when the
groupby() object is advanced, the previous group is no longer
visible. So, if that data is needed later, it should be stored as a
list:

(example follows)

So when you use:

print map(list, imap(itemgetter(1), gb))

each group is stored as a list *before* the groupby() object is
advanced, so that's ok.

However, when you use:

print map(list, map(itemgetter(1), gb))

the groupby() object is advanced to the end *before* each group is
stored as a list. According the the docs quoted above, this renders
each group no longer visible.

It is for the same reason that the following are different:

>>> def foo():
.... for i in range(5):
.... yield lambda: i
....
>>> map(lambda f: f(), imap(lambda x: x, foo()))
[0, 1, 2, 3, 4]
>>> map(lambda f: f(), map(lambda x: x, foo()))
[4, 4, 4, 4, 4]

Although looking back at it, I don't know if this will help you :)

[1] http://docs.python.org/library/itertools.html#itertoo...

--
Arnaud