[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

best practices: is collections.defaultdict my friend or not?

Pete Emerson

3/6/2010 1:22:00 AM

I've been wrestling with dicts. I hope at the very least what I
discovered helps someone else out, but I'm interested in hearing from
more learned python users.

I found out that adding a two dimensional element without defining
first dimension existing doesn't work:

>>> data = {}
>>> data['one']['two'] = 'three'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'one'
>>> data['one'] = {}
>>> data['one']['two'] = 'three'
>>> print data
{'one': {'two': 'three'}}

And through some research, I discovered collections.defaultdict (new
in Python 2.5, FWIW):

>>> import collections
>>> data = collections.defaultdict(dict)
>>> data['one']['two'] = 'three'
>>> print data
defaultdict(<type 'dict'>, {'one': {'two': 'three'}})

Why isn't the behavior of collections.defaultdict the default for a
dict?
Am I just revelling in my bad perl habits by not wanting to declare a
previous level first?
Is this sort of "more rigid" way of doing things common throughout
python, and is it best that I not fight it, but embrace it?

Your thoughts and comments are very much appreciated. I think my brain
already knows some of the answers, but my heart ... well, perl and I
go way back. Loving python so far, though.

Pete
11 Answers

Andreas Waldenburger

3/6/2010 2:03:00 AM

0

On Fri, 5 Mar 2010 17:22:14 -0800 (PST) Pete Emerson
<pemerson@gmail.com> wrote:

> I've been wrestling with dicts. I hope at the very least what I
> discovered helps someone else out, but I'm interested in hearing from
> more learned python users.
>
> I found out that adding a two dimensional element without defining
> first dimension existing doesn't work:
>
> >>> data = {}
> >>> data['one']['two'] = 'three'
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> KeyError: 'one'
Yeah, because that key 'one' indexes what exactly?


> [snip]
>
> And through some research, I discovered collections.defaultdict (new
> in Python 2.5, FWIW):
>
> [snip]
>
> Why isn't the behavior of collections.defaultdict the default for a
> dict?
Because explicit is better than implicit. How should Python know what
you want to fill your dict with? It could be anything that takes string
indices. Only two lines of code remove all the guesswork.


> Am I just revelling in my bad perl habits by not wanting to declare a
> previous level first?
Yes.

Python pretty much never guesses (not that I knew of anyway). I think
that is good, because it's just too easy to overlook some magic the
language does and spend an hour trying to understand why that Toyota
hit that wall.


> Is this sort of "more rigid" way of doing things common throughout
> python, and is it best that I not fight it, but embrace it?
>
Seriously, what answer do you expect? ;) When has fighting a language
ever brought about something really cool. (I'm really looking forward
to the geeky humorous replies this is bound to sprout.)



> Your thoughts and comments are very much appreciated. I think my brain
> already knows some of the answers, but my heart ... well, perl and I
> go way back. Loving python so far, though.
>
Welcome on board.

Have you typed "import this" in the command prompt yet?



--
INVALID? DE!

Andreas Waldenburger

3/6/2010 2:10:00 AM

0

On Fri, 5 Mar 2010 17:22:14 -0800 (PST) Pete Emerson
<pemerson@gmail.com> wrote:

> [snip]
> >>> data['one'] = {}
> >>> data['one']['two'] = 'three'
> >>> print data
> {'one': {'two': 'three'}}
>
> And through some research, I discovered collections.defaultdict (new
> in Python 2.5, FWIW):
>
> >>> import collections
> >>> data = collections.defaultdict(dict)
> >>> data['one']['two'] = 'three'
> >>> print data
> defaultdict(<type 'dict'>, {'one': {'two': 'three'}})
>
> [snip]
> Your thoughts and comments are very much appreciated. I think my brain
> already knows some of the answers, but my heart ... well, perl and I
> go way back. Loving python so far, though.
>

Oh, by the way: That defaultdict route is a pretty solid solution. Not
sure what problem you're trying to solve -- depending on your usecase,
there might be a better approach.

If you're just asking hypothetically and you're trying to apply a
Perl idiom to Python, there probably *is* a better solution.

/W

--
INVALID? DE!

MRAB

3/6/2010 2:27:00 AM

0

Pete Emerson wrote:
> I've been wrestling with dicts. I hope at the very least what I
> discovered helps someone else out, but I'm interested in hearing from
> more learned python users.
>
> I found out that adding a two dimensional element without defining
> first dimension existing doesn't work:
>
>>>> data = {}
>>>> data['one']['two'] = 'three'
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> KeyError: 'one'
>>>> data['one'] = {}
>>>> data['one']['two'] = 'three'
>>>> print data
> {'one': {'two': 'three'}}
>
> And through some research, I discovered collections.defaultdict (new
> in Python 2.5, FWIW):
>
>>>> import collections
>>>> data = collections.defaultdict(dict)
>>>> data['one']['two'] = 'three'
>>>> print data
> defaultdict(<type 'dict'>, {'one': {'two': 'three'}})
>
> Why isn't the behavior of collections.defaultdict the default for a
> dict?
> Am I just revelling in my bad perl habits by not wanting to declare a
> previous level first?
> Is this sort of "more rigid" way of doing things common throughout
> python, and is it best that I not fight it, but embrace it?
>
> Your thoughts and comments are very much appreciated. I think my brain
> already knows some of the answers, but my heart ... well, perl and I
> go way back. Loving python so far, though.
>
Someone once wrote about a case where he was porting a frontend from
Perl to Python. It called a backend and parsed the result. Sometimes
converting one of the fields to a number would raise a ValueError
because it would contain "ERR" instead of a number, which Perl, of
course, would silently convert to 0!

Python is all about refusing to guess, and complaining if there's an
error. :-)

Pete Emerson

3/6/2010 2:31:00 AM

0

On Mar 5, 6:10 pm, Andreas Waldenburger <use...@geekmail.INVALID>
wrote:
> On Fri, 5 Mar 2010 17:22:14 -0800 (PST) Pete Emerson
>
>
>
>
>
> <pemer...@gmail.com> wrote:
> > [snip]
> > >>> data['one'] = {}
> > >>> data['one']['two'] = 'three'
> > >>> print data
> > {'one': {'two': 'three'}}
>
> > And through some research, I discovered collections.defaultdict (new
> > in Python 2.5, FWIW):
>
> > >>> import collections
> > >>> data = collections.defaultdict(dict)
> > >>> data['one']['two'] = 'three'
> > >>> print data
> > defaultdict(<type 'dict'>, {'one': {'two': 'three'}})
>
> > [snip]
> > Your thoughts and comments are very much appreciated. I think my brain
> > already knows some of the answers, but my heart ... well, perl and I
> > go way back. Loving python so far, though.
>
> Oh, by the way: That defaultdict route is a pretty solid solution. Not
> sure what problem you're trying to solve -- depending on your usecase,
> there might be a better approach.
>
> If you're just asking hypothetically and you're trying to apply a
> Perl idiom to Python, there probably *is* a better solution.
>
> /W
>
> --
> INVALID? DE!

I found out about the need to declare the higher level as I was
reading in a JSON struct into a dict and then adding a new entry at a
lower level. Mostly just proof of concept stuff as I'm learning
python. I'm not sure that the use of defaultdict is really warranted
for me anywhere just yet. Mostly, I don't want to convert my perl to
python, that seems very counterproductive. Thank you very much for
your insight.

I was a little frightened of doing "import this" ("Hey, kid, run rm -
rf / and see what happens!"), but did, and the words are wise. :)

Pete

Paul Rubin

3/6/2010 2:54:00 AM

0

Pete Emerson <pemerson@gmail.com> writes:
> I found out that adding a two dimensional element without defining
> first dimension existing doesn't work:
>
>>>> data = {}
>>>> data['one']['two'] = 'three'

You can use a tuple as a subscript if you want:

data = {}
data['one','two'] = 'three'

Steven D'Aprano

3/6/2010 4:25:00 AM

0

On Fri, 05 Mar 2010 17:22:14 -0800, Pete Emerson wrote:

> Why isn't the behavior of collections.defaultdict the default for a
> dict?

Why would it be?

If you look up a key in a dict:

addressbook['Barney Rubble']

and you don't actually have Barney's address, should Python guess and
make something up?

In general, looking up a missing key is an error, and errors should never
pass silently unless explicitly silenced.

And for those cases where missing keys are not errors, you're spoiled for
choice:

dict.get
dict.setdefault
collections.defaultdict

try:
dict[key]
except KeyError:
do something else

Or even:

if key in dict:
dict[key]
else:
do something else




--
Steven

Pete Emerson

3/6/2010 5:37:00 AM

0

On Mar 5, 8:24 pm, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:
> On Fri, 05 Mar 2010 17:22:14 -0800, Pete Emerson wrote:
> > Why isn't the behavior of collections.defaultdict the default for a
> > dict?
>
> Why would it be?
>
> If you look up a key in a dict:
>
> addressbook['Barney Rubble']
>
> and you don't actually have Barney's address, should Python guess and
> make something up?
>
> In general, looking up a missing key is an error, and errors should never
> pass silently unless explicitly silenced.
>
> And for those cases where missing keys are not errors, you're spoiled for
> choice:
>
> dict.get
> dict.setdefault
> collections.defaultdict
>
> try:
>     dict[key]
> except KeyError:
>     do something else
>
> Or even:
>
> if key in dict:
>     dict[key]
> else:
>     do something else
>
> --
> Steven

My frame of reference for the past 10 < N < 15 years has been doing
this sort of assignment in perl:

$hash{key1}{key2} = value

I definitely agree that looking up a missing key should give an error.
The lazy perl programmer in me just wants to make an assignment to a
missing second key without defining the first key first. I'm not
saying it's right, I'm saying that it's something I'm trying to
unlearn, as I'm being convinced that it's the "best way" to do it in
python.

I'll take a look at dict.get and dict.setdefault, thank you for those.

I'm learning, and you're all very helpful, and I appreciate it!

Pete

Pete Emerson

3/6/2010 5:43:00 AM

0

On Mar 5, 6:26 pm, MRAB <pyt...@mrabarnett.plus.com> wrote:
> Pete Emerson wrote:
> > I've been wrestling with dicts. I hope at the very least what I
> > discovered helps someone else out, but I'm interested in hearing from
> > more learned python users.
>
> > I found out that adding a two dimensional element without defining
> > first dimension existing doesn't work:
>
> >>>> data = {}
> >>>> data['one']['two'] = 'three'
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> > KeyError: 'one'
> >>>> data['one'] = {}
> >>>> data['one']['two'] = 'three'
> >>>> print data
> > {'one': {'two': 'three'}}
>
> > And through some research, I discovered collections.defaultdict (new
> > in Python 2.5, FWIW):
>
> >>>> import collections
> >>>> data = collections.defaultdict(dict)
> >>>> data['one']['two'] = 'three'
> >>>> print data
> > defaultdict(<type 'dict'>, {'one': {'two': 'three'}})
>
> > Why isn't the behavior of collections.defaultdict the default for a
> > dict?
> > Am I just revelling in my bad perl habits by not wanting to declare a
> > previous level first?
> > Is this sort of "more rigid" way of doing things common throughout
> > python, and is it best that I not fight it, but embrace it?
>
> > Your thoughts and comments are very much appreciated. I think my brain
> > already knows some of the answers, but my heart ... well, perl and I
> > go way back. Loving python so far, though.
>
> Someone once wrote about a case where he was porting a frontend from
> Perl to Python. It called a backend and parsed the result. Sometimes
> converting one of the fields to a number would raise a ValueError
> because it would contain "ERR" instead of a number, which Perl, of
> course, would silently convert to 0!
>
> Python is all about refusing to guess, and complaining if there's an
> error. :-)

Perl is quite an amazing language, but it also definitely allows for
sloppy habits and poor coding behaviors, particularly for someone such
as me whose perl is completely self taught. I think that's why this
time around with python I'm trying to learn my prior experience and
seek help in areas where I suspect there is improvement to be made.
I'm really liking the rigid flexibility I'm experiencing with python
so far.

I'm thinking it's time for me to get a python reference or two, just
to kill a few trees. If anyone has any strong feelings about what
belongs in a "beginner but already learning quickly" library, please
toss them my way. I'll grep around the 'net for opinions on it, too.

Pete

Bruno Desthuilliers

3/6/2010 3:05:00 PM

0

Pete Emerson a écrit :
(snip)
> I'm really liking the rigid flexibility I'm experiencing with python
> so far.

"rigid flexibility" !-)

+1 QOTW - and welcome on board BTW.

Mark Lawrence

3/7/2010 9:33:00 AM

0

Pete Emerson wrote:
> On Mar 5, 6:10 pm, Andreas Waldenburger <use...@geekmail.INVALID>
> wrote:
>> On Fri, 5 Mar 2010 17:22:14 -0800 (PST) Pete Emerson
>>
>>
>>
>>
>>
>> <pemer...@gmail.com> wrote:
>>> [snip]
>>>>>> data['one'] = {}
>>>>>> data['one']['two'] = 'three'
>>>>>> print data
>>> {'one': {'two': 'three'}}
>>> And through some research, I discovered collections.defaultdict (new
>>> in Python 2.5, FWIW):
>>>>>> import collections
>>>>>> data = collections.defaultdict(dict)
>>>>>> data['one']['two'] = 'three'
>>>>>> print data
>>> defaultdict(<type 'dict'>, {'one': {'two': 'three'}})
>>> [snip]
>>> Your thoughts and comments are very much appreciated. I think my brain
>>> already knows some of the answers, but my heart ... well, perl and I
>>> go way back. Loving python so far, though.
>> Oh, by the way: That defaultdict route is a pretty solid solution. Not
>> sure what problem you're trying to solve -- depending on your usecase,
>> there might be a better approach.
>>
>> If you're just asking hypothetically and you're trying to apply a
>> Perl idiom to Python, there probably *is* a better solution.
>>
>> /W
>>
>> --
>> INVALID? DE!
>
> I found out about the need to declare the higher level as I was
> reading in a JSON struct into a dict and then adding a new entry at a
> lower level. Mostly just proof of concept stuff as I'm learning
> python. I'm not sure that the use of defaultdict is really warranted
> for me anywhere just yet. Mostly, I don't want to convert my perl to
> python, that seems very counterproductive. Thank you very much for
> your insight.
>
> I was a little frightened of doing "import this" ("Hey, kid, run rm -
> rf / and see what happens!"), but did, and the words are wise. :)
>
> Pete

After reading the words of wisdom try "import this" a second time and
watch what happens, it's quite interesting if you're not expecting the
output.

Mark Lawrence.