[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

loop over list and process into groups

Sneaky Wombat

3/4/2010 3:57:00 PM

[ {'vlan_or_intf': 'VLAN2021'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2022'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi7/33'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2051'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},
{'vlan_or_intf': 'VLAN2052'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},]

I want it to be converted to:

[{'2021':['Po1','Po306']},{'2022':['Gi7/33','Po1','Po306']},etc etc]

I was going to write a def to loop through and look for certain pre-
compiled regexs, and then put them in a new dictionary and append to a
list, but I'm having trouble thinking of a good way to capture each
dictionary. Each dictionary will have a key that is the vlan and the
value will be a list of interfaces that participate in that vlan.
Each list will be variable, many containing only one interface and
some containing many interfaces.

I thought about using itertools, but i only use that for fixed data.
I don't know of a good way to loop over variably sized data. I was
wondering if anyone had any ideas about a good way to convert this
list or dictionary into the right format that I need. The solution I
come up with will most likely be ugly and error prone, so I thought
i'd ask this python list while I work. Hopefully I learn a better way
to solve this problem.

Thanks!

I also have the data in a list,

[ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]
11 Answers

mk

3/4/2010 4:55:00 PM

0

Sneaky Wombat wrote:
> I was going to write a def to loop through and look for certain pre-
> compiled regexs, and then put them in a new dictionary and append to a
> list,

regexes are overkill in this case I think.

> [ 'VLAN4065',
> 'Interface',
> 'Gi9/6',
> 'Po2',
> 'Po3',
> 'Po306',
> 'VLAN4068',
> 'Interface',
> 'Gi9/6',
> 'VLAN4069',
> 'Interface',
> 'Gi9/6',]

Why not construct an intermediate dictionary?

elems = [ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]

def makeintermdict(elems):
vd = {}
vlan = None
for el in elems:
if el.startswith('VLAN'):
vlan = el.replace('VLAN','')
elif el == 'Interface':
vd[vlan] = []
else:
vd[vlan].append(el)
return vd

def makelist(interm):
finlist = []
for k in interm.keys():
finlist.append({k:interm[k]})
return finlist

if __name__ == "__main__":
intermediate = makeintermdict(elems)
print intermediate
finlist = makelist(intermediate)
print 'final', finlist


{'4068': ['Gi9/6'], '4069': ['Gi9/6'], '4065': ['Gi9/6', 'Po2', 'Po3',
'Po306']}
final [{'4068': ['Gi9/6']}, {'4069': ['Gi9/6']}, {'4065': ['Gi9/6',
'Po2', 'Po3', 'Po306']}]

I hope this is not your homework. :-)

Regards,
mk

Sneaky Wombat

3/4/2010 5:42:00 PM

0

On Mar 4, 10:55 am, mk <mrk...@gmail.com> wrote:
> Sneaky Wombat wrote:
> > I was going to write a def to loop through and look for certain pre-
> > compiled regexs, and then put them in a new dictionary and append to a
> > list,
>
> regexes are overkill in this case I think.
>
> > [ 'VLAN4065',
> >  'Interface',
> >  'Gi9/6',
> >  'Po2',
> >  'Po3',
> >  'Po306',
> >  'VLAN4068',
> >  'Interface',
> >  'Gi9/6',
> >  'VLAN4069',
> >  'Interface',
> >  'Gi9/6',]
>
> Why not construct an intermediate dictionary?
>
> elems = [ 'VLAN4065',
>   'Interface',
>   'Gi9/6',
>   'Po2',
>   'Po3',
>   'Po306',
>   'VLAN4068',
>   'Interface',
>   'Gi9/6',
>   'VLAN4069',
>   'Interface',
>   'Gi9/6',]
>
> def makeintermdict(elems):
>      vd = {}
>      vlan = None
>      for el in elems:
>          if el.startswith('VLAN'):
>              vlan = el.replace('VLAN','')
>          elif el == 'Interface':
>              vd[vlan] = []
>          else:
>              vd[vlan].append(el)
>      return vd
>
> def makelist(interm):
>      finlist = []
>      for k in interm.keys():
>          finlist.append({k:interm[k]})
>      return finlist
>
> if __name__ == "__main__":
>      intermediate = makeintermdict(elems)
>      print intermediate
>      finlist = makelist(intermediate)
>      print 'final', finlist
>
> {'4068': ['Gi9/6'], '4069': ['Gi9/6'], '4065': ['Gi9/6', 'Po2', 'Po3',
> 'Po306']}
> final [{'4068': ['Gi9/6']}, {'4069': ['Gi9/6']}, {'4065': ['Gi9/6',
> 'Po2', 'Po3', 'Po306']}]
>
> I hope this is not your homework. :-)
>
> Regards,
> mk

Thanks mk,

My approach was a lot more complex than yours, but your's is better.
I like itertools and was using islice to create tuples for (start,end)
string slicing. Too much work though. Thanks!

-j

lbolla

3/4/2010 5:59:00 PM

0

On Mar 4, 3:57 pm, Sneaky Wombat <joe.hr...@gmail.com> wrote:
> [ {'vlan_or_intf': 'VLAN2021'},
>  {'vlan_or_intf': 'Interface'},
>  {'vlan_or_intf': 'Po1'},
>  {'vlan_or_intf': 'Po306'},
>  {'vlan_or_intf': 'VLAN2022'},
>  {'vlan_or_intf': 'Interface'},
>  {'vlan_or_intf': 'Gi7/33'},
>  {'vlan_or_intf': 'Po1'},
>  {'vlan_or_intf': 'Po306'},
>  {'vlan_or_intf': 'VLAN2051'},
>  {'vlan_or_intf': 'Interface'},
>  {'vlan_or_intf': 'Gi9/6'},
>  {'vlan_or_intf': 'VLAN2052'},
>  {'vlan_or_intf': 'Interface'},
>  {'vlan_or_intf': 'Gi9/6'},]
>
> I want it to be converted to:
>
> [{'2021':['Po1','Po306']},{'2022':['Gi7/33','Po1','Po306']},etc etc]
>
> I was going to write a def to loop through and look for certain pre-
> compiled regexs, and then put them in a new dictionary and append to a
> list, but I'm having trouble thinking of a good way to capture each
> dictionary.  Each dictionary will have a key that is the vlan and the
> value will be a list of interfaces that participate in that vlan.
> Each list will be variable, many containing only one interface and
> some containing many interfaces.
>
> I thought about using itertools, but i only use that for fixed data.
> I don't know of a good way to loop over variably sized data.  I was
> wondering if anyone had any ideas about a good way to convert this
> list or dictionary into the right format that I need.  The solution I
> come up with will most likely be ugly and error prone, so I thought
> i'd ask this python list while I work.  Hopefully I learn a better way
> to solve this problem.
>
> Thanks!
>
> I also have the data in a list,
>
> [ 'VLAN4065',
>  'Interface',
>  'Gi9/6',
>  'Po2',
>  'Po3',
>  'Po306',
>  'VLAN4068',
>  'Interface',
>  'Gi9/6',
>  'VLAN4069',
>  'Interface',
>  'Gi9/6',]



===================================

from itertools import groupby

data = \
[ {'vlan_or_intf': 'VLAN2021'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2022'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi7/33'},
{'vlan_or_intf': 'Po1'},
{'vlan_or_intf': 'Po306'},
{'vlan_or_intf': 'VLAN2051'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},
{'vlan_or_intf': 'VLAN2052'},
{'vlan_or_intf': 'Interface'},
{'vlan_or_intf': 'Gi9/6'},]

def clean_up(lst):
return [d.values()[0] for d in data if d.values()[0] != 'Interface']

out = {}
for k, g in groupby(clean_up(data) , key=lambda s:
s.startswith('VLAN')):
if k:
key = list(g)[0].replace('VLAN','')
else:
out[key] = list(g)

print out
===================================

hth,
L.

nn

3/4/2010 7:19:00 PM

0



lbolla wrote:
> On Mar 4, 3:57 pm, Sneaky Wombat <joe.hr...@gmail.com> wrote:
> > [ {'vlan_or_intf': 'VLAN2021'},
> >  {'vlan_or_intf': 'Interface'},
> >  {'vlan_or_intf': 'Po1'},
> >  {'vlan_or_intf': 'Po306'},
> >  {'vlan_or_intf': 'VLAN2022'},
> >  {'vlan_or_intf': 'Interface'},
> >  {'vlan_or_intf': 'Gi7/33'},
> >  {'vlan_or_intf': 'Po1'},
> >  {'vlan_or_intf': 'Po306'},
> >  {'vlan_or_intf': 'VLAN2051'},
> >  {'vlan_or_intf': 'Interface'},
> >  {'vlan_or_intf': 'Gi9/6'},
> >  {'vlan_or_intf': 'VLAN2052'},
> >  {'vlan_or_intf': 'Interface'},
> >  {'vlan_or_intf': 'Gi9/6'},]
> >
> > I want it to be converted to:
> >
> > [{'2021':['Po1','Po306']},{'2022':['Gi7/33','Po1','Po306']},etc etc]
> >
> > I was going to write a def to loop through and look for certain pre-
> > compiled regexs, and then put them in a new dictionary and append to a
> > list, but I'm having trouble thinking of a good way to capture each
> > dictionary.  Each dictionary will have a key that is the vlan and the
> > value will be a list of interfaces that participate in that vlan.
> > Each list will be variable, many containing only one interface and
> > some containing many interfaces.
> >
> > I thought about using itertools, but i only use that for fixed data.
> > I don't know of a good way to loop over variably sized data.  I was
> > wondering if anyone had any ideas about a good way to convert this
> > list or dictionary into the right format that I need.  The solution I
> > come up with will most likely be ugly and error prone, so I thought
> > i'd ask this python list while I work.  Hopefully I learn a better way
> > to solve this problem.
> >
> > Thanks!
> >
> > I also have the data in a list,
> >
> > [ 'VLAN4065',
> >  'Interface',
> >  'Gi9/6',
> >  'Po2',
> >  'Po3',
> >  'Po306',
> >  'VLAN4068',
> >  'Interface',
> >  'Gi9/6',
> >  'VLAN4069',
> >  'Interface',
> >  'Gi9/6',]
>
>
>
> ===================================
>
> from itertools import groupby
>
> data = \
> [ {'vlan_or_intf': 'VLAN2021'},
> {'vlan_or_intf': 'Interface'},
> {'vlan_or_intf': 'Po1'},
> {'vlan_or_intf': 'Po306'},
> {'vlan_or_intf': 'VLAN2022'},
> {'vlan_or_intf': 'Interface'},
> {'vlan_or_intf': 'Gi7/33'},
> {'vlan_or_intf': 'Po1'},
> {'vlan_or_intf': 'Po306'},
> {'vlan_or_intf': 'VLAN2051'},
> {'vlan_or_intf': 'Interface'},
> {'vlan_or_intf': 'Gi9/6'},
> {'vlan_or_intf': 'VLAN2052'},
> {'vlan_or_intf': 'Interface'},
> {'vlan_or_intf': 'Gi9/6'},]
>
> def clean_up(lst):
> return [d.values()[0] for d in data if d.values()[0] != 'Interface']
>
> out = {}
> for k, g in groupby(clean_up(data) , key=lambda s:
> s.startswith('VLAN')):
> if k:
> key = list(g)[0].replace('VLAN','')
> else:
> out[key] = list(g)
>
> print out
> ===================================
>
> hth,
> L.

Good use of groupby. Here is what I ended up coming up:

from itertools import groupby
laninfo=[ 'VLAN4065',
'Interface',
'Gi9/6',
'Po2',
'Po3',
'Po306',
'VLAN4068',
'Interface',
'Gi9/6',
'VLAN4069',
'Interface',
'Gi9/6',]

def splitgrp(s, f=[False]):
f[0]^=s.startswith('VLAN')
return f[0]
lanlst=(list(g) for k,g in groupby(laninfo,key=splitgrp))
out={item[0][4:]:item[2:] for item in lanlst}
print(out)

Dennis Lee Bieber

3/5/2010 5:30:00 AM

0

On Thu, 4 Mar 2010 07:57:18 -0800 (PST), Sneaky Wombat
<joe.hrbek@gmail.com> declaimed the following in
gmane.comp.python.general:

> [ {'vlan_or_intf': 'VLAN2021'},
> {'vlan_or_intf': 'Interface'},
> {'vlan_or_intf': 'Po1'},
> {'vlan_or_intf': 'Po306'},
> {'vlan_or_intf': 'VLAN2022'},
> {'vlan_or_intf': 'Interface'},
> {'vlan_or_intf': 'Gi7/33'},
> {'vlan_or_intf': 'Po1'},
> {'vlan_or_intf': 'Po306'},
> {'vlan_or_intf': 'VLAN2051'},
> {'vlan_or_intf': 'Interface'},
> {'vlan_or_intf': 'Gi9/6'},
> {'vlan_or_intf': 'VLAN2052'},
> {'vlan_or_intf': 'Interface'},
> {'vlan_or_intf': 'Gi9/6'},]
>
That is an interesting data set...

It is a list of ONE ELEMENT dictionaries...
> I want it to be converted to:
>
> [{'2021':['Po1','Po306']},{'2022':['Gi7/33','Po1','Po306']},etc etc]

And so is your specified output... How do you intend to use this
data structure? Compared to a dictionary with multiple entries:

{ "2021" : ["P01", "Po306"],
"2022" : ["Gi7/33", "Po1", "Po306"],
.... }

which can be used via keys: converted["2022"] would retrieve the list of
whatever's...
> I also have the data in a list,
>
> [ 'VLAN4065',
> 'Interface',
> 'Gi9/6',
> 'Po2',
> 'Po3',
> 'Po306',
> 'VLAN4068',
> 'Interface',
> 'Gi9/6',
> 'VLAN4069',
> 'Interface',
> 'Gi9/6',]

Offhand, the "Interface" entries are superfluous. You appear to be
looking for items starting with VLAN, stripping off the VLAN and using
the remaining numbers as the "name", and then collecting all entries up
to the next "VLAN...", ignoring the "Interface" item.

-=-=-=-=-=-=-=-=-=-=-=-=-
INDATA = [ "VLAN4065",
"Interface",
"Gi9/6",
"Po2",
"Po3",
"Po306",
"VLAN4068",
"Interface",
"Gi9/6",
"VLAN4069",
"Interface",
"Gi9/6" ]

outDict = {}

key = None
for item in INDATA:
if item.startswith("VLAN"):
if key:
outDict[key] = items
key = item[4:]
items = []
elif item == "Interface":
pass
else:
if key: #if no key, can't use data
items.append(item)
if key: #handle last set
outDict[key] = items

from pprint import pprint
pprint(outDict)
-=-=-=-=-=-=-=-=-=-=-=-=-
{'4065': ['Gi9/6', 'Po2', 'Po3', 'Po306'],
'4068': ['Gi9/6'],
'4069': ['Gi9/6']}
-=-=-=-=-=-=-=-=-=-=-=-=-


--
Wulfraed Dennis Lee Bieber KD6MOG
wlfraed@ix.netcom.com HTTP://wlfraed.home.netcom.com/

Paul Rubin

3/5/2010 9:46:00 AM

0

lbolla <lbolla@gmail.com> writes:
> for k, g in groupby(clean_up(data) , key=lambda s: s.startswith('VLAN')):
> if k:
> key = list(g)[0].replace('VLAN','')

This is the nicest solution, I think. Mine was more cumbersome.

mk

3/5/2010 1:26:00 PM

0

Sneaky Wombat wrote:
> [ 'VLAN4065',
> 'Interface',
> 'Gi9/6',
> 'Po2',
> 'Po3',
> 'Po306',
> 'VLAN4068',
> 'Interface',
> 'Gi9/6',
> 'VLAN4069',
> 'Interface',
> 'Gi9/6',]

Hey, I just invented a cute ;-) two-liner using list comprehensions:

# alist = list above

tmp, dk = [], {}
[(x.startswith('VLAN') and (dk.setdefault(x,[]) or tmp.append(x))) or
(not x.startswith('VLAN') and dk[tmp[-1]].append(x)) for x in alist
if x != 'Interface']

No need to use a nuke like itertools to kill a fly. ;-)

Regards,
mk

nn

3/5/2010 4:41:00 PM

0



mk wrote:
> Sneaky Wombat wrote:
> > [ 'VLAN4065',
> > 'Interface',
> > 'Gi9/6',
> > 'Po2',
> > 'Po3',
> > 'Po306',
> > 'VLAN4068',
> > 'Interface',
> > 'Gi9/6',
> > 'VLAN4069',
> > 'Interface',
> > 'Gi9/6',]
>
> Hey, I just invented a cute ;-) two-liner using list comprehensions:
>
> # alist = list above
>
> tmp, dk = [], {}
> [(x.startswith('VLAN') and (dk.setdefault(x,[]) or tmp.append(x))) or
> (not x.startswith('VLAN') and dk[tmp[-1]].append(x)) for x in alist
> if x != 'Interface']
>
> No need to use a nuke like itertools to kill a fly. ;-)
>
> Regards,
> mk

Oh my! You could have at least used some "if else" to make it a little
bit easier on the eyes :-)

[(dk.setdefault(x,[]) or tmp.append(x))
if x.startswith('VLAN')
else dk[tmp[-1]].append(x)
for x in alist
if x != 'Interface']

lbolla

3/5/2010 5:09:00 PM

0

On Mar 5, 1:26 pm, mk <mrk...@gmail.com> wrote:
> Sneaky Wombat wrote:
> > [ 'VLAN4065',
> >  'Interface',
> >  'Gi9/6',
> >  'Po2',
> >  'Po3',
> >  'Po306',
> >  'VLAN4068',
> >  'Interface',
> >  'Gi9/6',
> >  'VLAN4069',
> >  'Interface',
> >  'Gi9/6',]
>
> Hey, I just invented a cute ;-) two-liner using list comprehensions:
>
> # alist = list above
>
> tmp, dk = [], {}
> [(x.startswith('VLAN') and (dk.setdefault(x,[]) or tmp.append(x))) or
> (not x.startswith('VLAN') and dk[tmp[-1]].append(x))    for x in alist
> if x != 'Interface']
>
> No need to use a nuke like itertools to kill a fly. ;-)
>
> Regards,
> mk

It looks like Perl ;-)

mk

3/5/2010 5:11:00 PM

0

nn wrote:

> Oh my! You could have at least used some "if else" to make it a little
> bit easier on the eyes :-)

That's my entry into """'Obfuscated' "Python" '"''code''"'
'"contest"'""" and I'm proud of it. ;-)

Regards,
mk