[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

Using a dict as if it were a module namespace

Steven D'Aprano

1/27/2008 7:45:00 AM

I have a problem which I think could be solved by using a dict as a
namespace, in a similar way that exec and eval do.

When using the timeit module, it is very inconvenient to have to define
functions as strings. A good alternative is to create the function as
normal, and import it:

def myfunc(x, y):
return x+y

timeit.Timer("myfunc(59, 60)", "from __main__ import myfunc").timeit()


Not only is this an easy idiom to follow, but myfunc can live in another
module: just replace __main__ with the module name.

Now, I'm trying to build a suite of tests to use with timeit. I have a
bunch of tests which I've created as dicts:

test_suite= [dict(x=59, y=60), dict(x=-1, y=-2)]

What I *think* I want to do is use the from ... import idiom to grab
arguments from the dicts as if they were modules, but this doesn't work:

expr = "myfunc(x, y)"
for test in test_suite:
setup = "from __main__ import myfunc; from test import x, y"
t = timeit.Timer(expr, setup).timeit()


Even if the Timer could see test, it is not a module and you can't import
from it. Naturally.


Alternatives that I have found:

(1) Import the test and grab the values needed from it:

setup = """from __main__ import myfunc, test
x, y = test['x'], test['y']"""


I don't like this one. It doesn't seem very elegant to me, and it gets
unwieldy as the complexity increases. Every item I need from test has to
be named twice, violating the principle Don't Repeat Yourself. If the
tests change, the setup string has to be explicitly changed also.


(2) Mess with the global namespace:

globals().update(t)
setup = "from __main__ import myfunc"

I don't like this one. It looks hackish, and I worry about conflicts and
side-effects. If it works (and I haven't tested it) it relies on an
implementation detail of timeit.Timer.__init__, namely the line
"exec code in globals(), ns". Worst of all, it pollutes or even mangles
the global namespace of the calling code, not the code being tested.


(3) Explicitly pass a namespace dict to the Timer class, possibly even
getting rid of setup altogether:

test['myfunc'] = myfunc
t = timeit.Timer(expr, '', ns=test).timeit()

This would be the most elegant solution, but at this time it is
completely hypothetical. Timer does not have that functionality.


(4) Dump the test data straight into the setup string:

setup = "from __main__ import myfunc; x = %(x)s; y = %(y)s" % t

Again, unwieldy and against DRY. The additional disadvantage is that
there are many types of test data that can't be converted to and from
strings like that.


What do others think? Have I missed something? What other alternatives
are there?



--
Steven
5 Answers

thebjorn

1/27/2008 8:46:00 AM

0

On Jan 27, 8:45 am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:
> I have a problem which I think could be solved by using a dict as a
> namespace, in a similar way that exec and eval do.
>
> When using the timeit module, it is very inconvenient to have to define
> functions as strings. A good alternative is to create the function as
> normal, and import it:
>
> def myfunc(x, y):
> return x+y
>
> timeit.Timer("myfunc(59, 60)", "from __main__ import myfunc").timeit()
>
> Not only is this an easy idiom to follow, but myfunc can live in another
> module: just replace __main__ with the module name.
>
> Now, I'm trying to build a suite of tests to use with timeit. I have a
> bunch of tests which I've created as dicts:
>
> test_suite= [dict(x=59, y=60), dict(x=-1, y=-2)]
>
> What I *think* I want to do is use the from ... import idiom to grab
> arguments from the dicts as if they were modules, but this doesn't work:
>
> expr = "myfunc(x, y)"
> for test in test_suite:
> setup = "from __main__ import myfunc; from test import x, y"
> t = timeit.Timer(expr, setup).timeit()
>
> Even if the Timer could see test, it is not a module and you can't import
> from it. Naturally.
>
> Alternatives that I have found:
>
> (1) Import the test and grab the values needed from it:
>
> setup = """from __main__ import myfunc, test
> x, y = test['x'], test['y']"""
>
> I don't like this one. It doesn't seem very elegant to me, and it gets
> unwieldy as the complexity increases. Every item I need from test has to
> be named twice, violating the principle Don't Repeat Yourself. If the
> tests change, the setup string has to be explicitly changed also.
>
> (2) Mess with the global namespace:
>
> globals().update(t)
> setup = "from __main__ import myfunc"
>
> I don't like this one. It looks hackish, and I worry about conflicts and
> side-effects. If it works (and I haven't tested it) it relies on an
> implementation detail of timeit.Timer.__init__, namely the line
> "exec code in globals(), ns". Worst of all, it pollutes or even mangles
> the global namespace of the calling code, not the code being tested.
>
> (3) Explicitly pass a namespace dict to the Timer class, possibly even
> getting rid of setup altogether:
>
> test['myfunc'] = myfunc
> t = timeit.Timer(expr, '', ns=test).timeit()
>
> This would be the most elegant solution, but at this time it is
> completely hypothetical. Timer does not have that functionality.
>
> (4) Dump the test data straight into the setup string:
>
> setup = "from __main__ import myfunc; x = %(x)s; y = %(y)s" % t
>
> Again, unwieldy and against DRY. The additional disadvantage is that
> there are many types of test data that can't be converted to and from
> strings like that.
>
> What do others think? Have I missed something? What other alternatives
> are there?
>
> --
> Steven

You might have lost me, but wouldn't it be easier to do some variation
on this

test_suite = [
'(x=59, y=60)', # either have strings here...
'(x=-1, y=-2)',
]

for test in test_suite:
# ... or convert the dicts to appropriate strings here...
expr = 'myfunc' + test
t = timeit.Timer(expr, 'from __main__ import myfunc').timeit()
...

-- bjorn

Ross Ridge

1/27/2008 3:04:00 PM

0

Steven D'Aprano writes:
>(1) Import the test and grab the values needed from it:
>
> setup = """from __main__ import myfunc, test
>x, y = test['x'], test['y']"""
>
>
>I don't like this one. It doesn't seem very elegant to me, and it gets
>unwieldy as the complexity increases. Every item I need from test has to
>be named twice, violating the principle Don't Repeat Yourself. If the
>tests change, the setup string has to be explicitly changed also.

I think this is the way to go as it follows the principle of "say what
you mean." You can however simplify it, and repeat yourself less,
by using the extended call syntax:

expr = "myfunc(**test)"
setup = """from __main__ import myfunc, test"""

....
>I don't like this one. It looks hackish, and I worry about conflicts and
>side-effects. If it works (and I haven't tested it) it relies on an
>implementation detail of timeit.Timer.__init__, namely the line
>"exec code in globals(), ns". Worst of all, it pollutes or even mangles
>the global namespace of the calling code, not the code being tested.

It wouldn't work because the timeit module's "globals" are different
from the __main__ module's globals.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rridge@csclub.uwaterloo.ca
-()-/()/ http://www.csclub.uwaterloo.c...
db //

Steven Bethard

1/27/2008 6:16:00 PM

0

Steven D'Aprano wrote:
> I have a problem which I think could be solved by using a dict as a
> namespace, in a similar way that exec and eval do.
>
> When using the timeit module, it is very inconvenient to have to define
> functions as strings. A good alternative is to create the function as
> normal, and import it:
>
> def myfunc(x, y):
> return x+y
>
> timeit.Timer("myfunc(59, 60)", "from __main__ import myfunc").timeit()
>
>
> Not only is this an easy idiom to follow, but myfunc can live in another
> module: just replace __main__ with the module name.
>
> Now, I'm trying to build a suite of tests to use with timeit. I have a
> bunch of tests which I've created as dicts:
>
> test_suite= [dict(x=59, y=60), dict(x=-1, y=-2)]
>
> What I *think* I want to do is use the from ... import idiom to grab
> arguments from the dicts as if they were modules, but this doesn't work:
>
> expr = "myfunc(x, y)"
> for test in test_suite:
> setup = "from __main__ import myfunc; from test import x, y"
> t = timeit.Timer(expr, setup).timeit()
[snip]
> (2) Mess with the global namespace:
>
> globals().update(t)
> setup = "from __main__ import myfunc"

Why not mess with the namespace inside the setup code? E.g.::

>>> test_suite = [dict(x=59, y=60), dict(x=-1, y=-2)]
>>> expr = "myfunc(x, y)"
>>> for test in test_suite:
... setup = textwrap.dedent('''
... from __main__ import myfunc, test
... globals().update(test)''')
... t = timeit.Timer(expr, setup).timeit()
...

That shouldn't pollute your actual namespace::

>>> x
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
NameError: name 'x' is not defined
>>> y
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
NameError: name 'y' is not defined

STeVe

Steven D'Aprano

1/28/2008 7:52:00 AM

0

On Sun, 27 Jan 2008 11:16:00 -0700, Steven Bethard wrote:

> Why not mess with the namespace inside the setup code? E.g.::
[snip]

Ah, thanks, good thinking. It's not perfect, but it should be close
enough for what I'm trying to do.


--
Steven
who still wishes you could explicitly pass a namespace to timeit.Timer...

Steven D'Aprano

1/28/2008 8:10:00 AM

0

On Sun, 27 Jan 2008 10:04:06 -0500, Ross Ridge wrote:

> I think this is the way to go as it follows the principle of "say what
> you mean." You can however simplify it, and repeat yourself less, by
> using the extended call syntax:
>
> expr = "myfunc(**test)"
> setup = """from __main__ import myfunc, test"""

Ah, also good thinking! I'd have to modify it, because the tests carry
other information which shouldn't be passed to the test function, but
that's a simple modification.

Thanks to all who replied.


--
Steven