[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

Python Generators

mpc

3/15/2008 8:35:00 PM

HI all,
I am trying to write a while loop that will iterate over generators to
capture all the headers of FFCache directories. However, the
generators embedded within the argument of another generator do not
seem to re-initiate. the example below loops through and initiates the
generator embedded in the argument only once. Can anyone explain while
the generator will not re-initiate, and suggest a simple fix?

#!/usr/bin/env python
import os,struct,time
def generate_headers(cachedir):
for name, blocksize in cachefiles:
pathname = os.path.join(cachedir,name)
f = open(pathname,"rb")
f.seek(4096)
while True:
header = f.read(36)
if not header: break
fields = struct.unpack(">9I",header)
if fields[0] == 0x00010008: yield f, fields
fp = f.tell()
offset = fp % blocksize
if offset:
f.seek(blocksize - offset,1)
f.close()
def concatenate(sequences):
for seq in sequences:
for item in seq:
yield item

all_caches = (path for path,dirlist,filelist in os.walk("/Users/") if
'_CACHE_MAP_' in filelist)
cachefiles = [('_CACHE_001_',256),('_CACHE_002_',1024),('_CACHE_003_',
4096)]
n = 0
while True:
n += 1
time.sleep(0.5)
headers = concatenate(generate_headers(path) for path in
all_caches)
for h in headers:
print h,n
# this doesn't work either

while True:
n += 1
time.sleep(0.5)
for path in all_caches:
headers = generate_headers(path)
for h in headers:
print h,n
# however if I hard code the path
path = "FFCache"
while True:
n += 1
headers = generate_headers(path)
for h in headers:
print h,n
#but of course i do not wish to hard code the path.
4 Answers

sturlamolden

3/15/2008 9:26:00 PM

0

On 15 Mar, 21:35, mpc <mcoh...@gmail.com> wrote:


> generator embedded in the argument only once. Can anyone explain while
> the generator will not re-initiate, and suggest a simple fix?


I am not sure what you are trying to do, but it seems a bit confused.


>>> def concat(seq):
for s in seq: yield s

>>> seq = xrange(3)

>>> for n in xrange(5):
h = concat(s for s in seq)
for i in h: print i,n


0 0
1 0
2 0
0 1
1 1
2 1
0 2
1 2
2 2
0 3
1 3
2 3
0 4
1 4
2 4

Generators work they way they should. Even when one is used as
argument for another.








Peter Otten

3/15/2008 10:07:00 PM

0

mpc wrote:

> I am trying to write a while loop that will iterate over generators to
> capture all the headers of FFCache directories. However, the
> generators embedded within the argument of another generator do not
> seem to re-initiate. the example below loops through and initiates the
> generator embedded in the argument only once. Can anyone explain while
> the generator will not re-initiate, and suggest a simple fix?

A generator or generator expression does indeed only run once.

>>> for i in range(3):
.... print "---", i, "---"
.... for k in gen: print k,
.... print
....
--- 0 ---
1 3
--- 1 ---

--- 2 ---

The fix is to use a list or list comprehension, or make a new generator
every time you need one:

>>> for i in range(3):
.... print "---", i, "---"
.... gen = (i for i in range(5) if i%2)
.... for k in gen: print k,
.... print
....
--- 0 ---
1 3
--- 1 ---
1 3
--- 2 ---
1 3

At first glance I would guess that in your case all_caches is the culprit
that has to be moved into the 'while True: ...' loop.

> while True:
> n += 1
> time.sleep(0.5)

all_caches = (path for path,dirlist,filelist in os.walk("/Users/") if
'_CACHE_MAP_' in filelist)

> for path in all_caches:
> headers = generate_headers(path)
> for h in headers:
> print h,n

(Untested, because you didn't bother to provide a self-contained example)

Peter

Peter Otten

3/15/2008 10:10:00 PM

0

mpc wrote:

> I am trying to write a while loop that will iterate over generators to
> capture all the headers of FFCache directories. However, the
> generators embedded within the argument of another generator do not
> seem to re-initiate. the example below loops through and initiates the
> generator embedded in the argument only once. Can anyone explain while
> the generator will not re-initiate, and suggest a simple fix?

A generator or generator expression does indeed only run once.

>>> gen = (i for i in range(5) if i%2)
>>> for i in range(3):
.... print "---", i, "---"
.... for k in gen: print k,
.... print
....
--- 0 ---
1 3
--- 1 ---

--- 2 ---

The fix is to use a list or list comprehension, or make a new generator
every time you need one:

>>> for i in range(3):
.... print "---", i, "---"
.... gen = (i for i in range(5) if i%2)
.... for k in gen: print k,
.... print
....
--- 0 ---
1 3
--- 1 ---
1 3
--- 2 ---
1 3

At first glance I would guess that in your case all_caches is the culprit
that has to be moved into the 'while True: ...' loop.

> while True:
> n += 1
> time.sleep(0.5)

all_caches = (path for path,dirlist,filelist in os.walk("/Users/") if
'_CACHE_MAP_' in filelist)

> for path in all_caches:
> headers = generate_headers(path)
> for h in headers:
> print h,n

(Untested, because you didn't bother to provide a self-contained example)

Peter

Matt Nordhoff

3/16/2008 7:25:00 AM

0

mpc wrote:

<snip>

> def concatenate(sequences):
> for seq in sequences:
> for item in seq:
> yield item

You should check out itertools.chain(). It does this. You call it like
"chain(seq1, seq2, ...)" instead of "chain(sequences)" though, which may
be a problem for you.

The rest of itertools might be interesting too:

<http://docs.python.org/lib/module-itertool...

<snip>
--