[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.python

A "scopeguard" for Python

Alf P. Steinbach

3/3/2010 3:10:00 PM

For C++ Petru Marginean once invented the "scope guard" technique (elaborated on
by Andrei Alexandrescu, they published an article about it in DDJ) where all you
need to do to ensure some desired cleanup at the end of a scope, even when the
scope is exited via an exception, is to declare a ScopeGuard w/desired action.

The C++ ScopeGuard was/is for those situations where you don't have proper
classes with automatic cleanup, which happily is seldom the case in good C++
code, but languages like Java and Python don't support automatic cleanup and so
the use case for something like ScopeGuard is ever present.

For use with a 'with' statement and possibly suitable 'lambda' arguments:


<code>
class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( is_callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" ) from
</code>


I guess the typical usage would be what I used it for, a case where the cleanup
action (namely, changing back to an original directory) apparently didn't fit
the standard library's support for 'with', like

with Cleanup as at_cleanup:
# blah blah
chdir( somewhere )
at_cleanup.call( lambda: chdir( original_dir ) )
# blah blah

Another use case might be where one otherwise would get into very deep nesting
of 'with' statements with every nested 'with' at the end, like a degenerate tree
that for all purposes is a list. Then the above, or some variant, can help to
/flatten/ the structure. To get rid of that silly & annoying nesting. :-)


Cheers,

- Alf (just sharing, it's not seriously tested code)
53 Answers

Mike Kent

3/3/2010 3:39:00 PM

0

What's the compelling use case for this vs. a simple try/finally?

original_dir = os.getcwd()
try:
os.chdir(somewhere)
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup

Alf P. Steinbach

3/3/2010 3:56:00 PM

0

* Mike Kent:
> What's the compelling use case for this vs. a simple try/finally?

if you thought about it you would mean a simple "try/else". "finally" is always
executed. which is incorrect for cleanup

by the way, that's one advantage:

a "with Cleanup" is difficult to get wrong, while a "try" is easy to get wrong,
as you did here


---

another general advantage is as for the 'with' statement generally



> original_dir = os.getcwd()
> try:
> os.chdir(somewhere)
> # Do other stuff

also, the "do other stuff" can be a lot of code

and also, with more than one action the try-else introduces a lot of nesting


> finally:
> os.chdir(original_dir)
> # Do other cleanup


cheers & hth.,

- alf

Robert Kern

3/3/2010 5:00:00 PM

0

On 2010-03-03 09:39 AM, Mike Kent wrote:
> What's the compelling use case for this vs. a simple try/finally?
>
> original_dir = os.getcwd()
> try:
> os.chdir(somewhere)
> # Do other stuff
> finally:
> os.chdir(original_dir)
> # Do other cleanup

A custom-written context manager looks nicer and can be more readable.

from contextlib import contextmanager
import os

@contextmanager
def pushd(path):
original_dir = os.getcwd()
os.chdir(path)
try:
yield
finally:
os.chdir(original_dir)


with pushd(somewhere):
...


I don't think a general purpose ScopeGuard context manager has any such benefits
over the try: finally:, though.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Robert Kern

3/3/2010 5:09:00 PM

0

On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
> * Mike Kent:
>> What's the compelling use case for this vs. a simple try/finally?
>
> if you thought about it you would mean a simple "try/else". "finally" is
> always executed. which is incorrect for cleanup

Eh? Failed execution doesn't require cleanup? The example you gave is definitely
equivalent to the try: finally: that Mike posted. The actions are always
executed in your example, not just when an exception isn't raised.

From your post, the scope guard technique is used "to ensure some desired
cleanup at the end of a scope, even when the scope is exited via an exception."
This is precisely what the try: finally: syntax is for. The with statement
allows you to encapsulate repetitive boilerplate into context managers, but a
general purpose context manager like your Cleanup class doesn't take advantage
of this.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Alf P. Steinbach

3/3/2010 5:10:00 PM

0

* Robert Kern:
> On 2010-03-03 09:39 AM, Mike Kent wrote:
>> What's the compelling use case for this vs. a simple try/finally?
>>
>> original_dir = os.getcwd()
>> try:
>> os.chdir(somewhere)
>> # Do other stuff
>> finally:
>> os.chdir(original_dir)
>> # Do other cleanup
>
> A custom-written context manager looks nicer and can be more readable.
>
> from contextlib import contextmanager
> import os
>
> @contextmanager
> def pushd(path):
> original_dir = os.getcwd()
> os.chdir(path)
> try:
> yield
> finally:
> os.chdir(original_dir)
>
>
> with pushd(somewhere):
> ...
>
>
> I don't think a general purpose ScopeGuard context manager has any such
> benefits over the try: finally:, though.

I don't think that's a matter of opinion, since one is correct while the other
is incorrect.


Cheers,

- ALf

Alf P. Steinbach

3/3/2010 5:19:00 PM

0

* Robert Kern:
> On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
>> * Mike Kent:
>>> What's the compelling use case for this vs. a simple try/finally?
>>
>> if you thought about it you would mean a simple "try/else". "finally" is
>> always executed. which is incorrect for cleanup
>
> Eh? Failed execution doesn't require cleanup? The example you gave is
> definitely equivalent to the try: finally: that Mike posted.

Sorry, that's incorrect: it's not.

With correct code (mine) cleanup for action A is only performed when action A
succeeds.

With incorrect code cleanup for action A is performed when A fails.


> The actions
> are always executed in your example,

Sorry, that's incorrect.


> [The actions are] not [executed] just when an exception isn't raised.

Sorry, that's incorrect.


> From your post, the scope guard technique is used "to ensure some
> desired cleanup at the end of a scope, even when the scope is exited via
> an exception." This is precisely what the try: finally: syntax is for.

You'd have to nest it. That's ugly. And more importantly, now two people in this
thread (namely you and Mike) have demonstrated that they do not grok the try
functionality and manage to write incorrect code, even arguing that it's correct
when informed that it's not, so it's a pretty fragile construct, like goto.


> The with statement allows you to encapsulate repetitive boilerplate into
> context managers, but a general purpose context manager like your
> Cleanup class doesn't take advantage of this.

I'm sorry but that's pretty meaningless. It's like: "A house allows you to
encapsulate a lot of stinking garbage, but your house doesn't take advantage of
that, it's disgustingly clean". Hello.


Cheers & hth.,

- Alf

Robert Kern

3/3/2010 6:52:00 PM

0

On 2010-03-03 11:18 AM, Alf P. Steinbach wrote:
> * Robert Kern:
>> On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
>>> * Mike Kent:
>>>> What's the compelling use case for this vs. a simple try/finally?
>>>
>>> if you thought about it you would mean a simple "try/else". "finally" is
>>> always executed. which is incorrect for cleanup
>>
>> Eh? Failed execution doesn't require cleanup? The example you gave is
>> definitely equivalent to the try: finally: that Mike posted.
>
> Sorry, that's incorrect: it's not.
>
> With correct code (mine) cleanup for action A is only performed when
> action A succeeds.
>
> With incorrect code cleanup for action A is performed when A fails.

Oh?

$ cat cleanup.py

class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" )

def print_(x):
print x

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup executed without an exception."))

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
raise RuntimeError()

$ python cleanup.py
Cleanup executed without an exception.
Cleanup execute with an exception.
Traceback (most recent call last):
File "cleanup.py", line 28, in <module>
raise RuntimeError()
RuntimeError

>> The actions are always executed in your example,
>
> Sorry, that's incorrect.

Looks like it to me.

>> From your post, the scope guard technique is used "to ensure some
>> desired cleanup at the end of a scope, even when the scope is exited
>> via an exception." This is precisely what the try: finally: syntax is
>> for.
>
> You'd have to nest it. That's ugly. And more importantly, now two people
> in this thread (namely you and Mike) have demonstrated that they do not
> grok the try functionality and manage to write incorrect code, even
> arguing that it's correct when informed that it's not, so it's a pretty
> fragile construct, like goto.

Uh-huh.

>> The with statement allows you to encapsulate repetitive boilerplate
>> into context managers, but a general purpose context manager like your
>> Cleanup class doesn't take advantage of this.
>
> I'm sorry but that's pretty meaningless. It's like: "A house allows you
> to encapsulate a lot of stinking garbage, but your house doesn't take
> advantage of that, it's disgustingly clean". Hello.

No, I'm saying that your Cleanup class is about as ugly as the try: finally:. It
just shifts the ugliness around. There is a way to use the with statement to
make things look better and more readable in certain situations, namely where
there is some boilerplate that you would otherwise repeat in many places using
try: finally:. You can encapsulate that repetitive code into a class or a
@contextmanager generator and just call the contextmanager. A generic context
manager where you register callables doesn't replace any boilerplate. You still
repeat all of the cleanup code everywhere. What's more, because you have to
shove everything into a callable, you have significantly less flexibility than
the try: finally:.

I will admit that you can put the cleanup code closer to the code that needs to
get cleaned up, but you pay a price for that.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Alf P. Steinbach

3/3/2010 7:32:00 PM

0

* Robert Kern:
> On 2010-03-03 11:18 AM, Alf P. Steinbach wrote:
>> * Robert Kern:
>>> On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
>>>> * Mike Kent:
>>>>> What's the compelling use case for this vs. a simple try/finally?
>>>>
>>>> if you thought about it you would mean a simple "try/else".
>>>> "finally" is
>>>> always executed. which is incorrect for cleanup
>>>
>>> Eh? Failed execution doesn't require cleanup? The example you gave is
>>> definitely equivalent to the try: finally: that Mike posted.
>>
>> Sorry, that's incorrect: it's not.
>>
>> With correct code (mine) cleanup for action A is only performed when
>> action A succeeds.
>>
>> With incorrect code cleanup for action A is performed when A fails.
>
> Oh?
>
> $ cat cleanup.py
>
> class Cleanup:
> def __init__( self ):
> self._actions = []
>
> def call( self, action ):
> assert( callable( action ) )
> self._actions.append( action )
>
> def __enter__( self ):
> return self
>
> def __exit__( self, x_type, x_value, x_traceback ):
> while( len( self._actions ) != 0 ):
> try:
> self._actions.pop()()
> except BaseException as x:
> raise AssertionError( "Cleanup: exception during cleanup" )
>
> def print_(x):
> print x
>
> with Cleanup() as at_cleanup:
> at_cleanup.call(lambda: print_("Cleanup executed without an
> exception."))
>
> with Cleanup() as at_cleanup:

*Here* is where you should

1) Perform the action for which cleanup is needed.

2) Let it fail by raising an exception.


> at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
> raise RuntimeError()

With an exception raised here cleanup should of course be performed.

And just in case you didn't notice: the above is not a test of the example I gave.


> $ python cleanup.py
> Cleanup executed without an exception.
> Cleanup execute with an exception.
> Traceback (most recent call last):
> File "cleanup.py", line 28, in <module>
> raise RuntimeError()
> RuntimeError
>
>>> The actions are always executed in your example,
>>
>> Sorry, that's incorrect.
>
> Looks like it to me.

I'm sorry, but you're

1) not testing my example which you're claiming that you're testing, and

2) not even showing anything about your earlier statements, which were
just incorrect.

You're instead showing that my code works as it should for the case that you're
testing, which is a bit unnecessary since I knew that, but thanks anyway.

I'm not sure what that shows, except that you haven't grokked this yet.


>>> From your post, the scope guard technique is used "to ensure some
>>> desired cleanup at the end of a scope, even when the scope is exited
>>> via an exception." This is precisely what the try: finally: syntax is
>>> for.
>>
>> You'd have to nest it. That's ugly. And more importantly, now two people
>> in this thread (namely you and Mike) have demonstrated that they do not
>> grok the try functionality and manage to write incorrect code, even
>> arguing that it's correct when informed that it's not, so it's a pretty
>> fragile construct, like goto.
>
> Uh-huh.

Yeah. Consider that you're now for the third time failing to grasp the concept
of cleanup for a successful operation.


>>> The with statement allows you to encapsulate repetitive boilerplate
>>> into context managers, but a general purpose context manager like your
>>> Cleanup class doesn't take advantage of this.
>>
>> I'm sorry but that's pretty meaningless. It's like: "A house allows you
>> to encapsulate a lot of stinking garbage, but your house doesn't take
>> advantage of that, it's disgustingly clean". Hello.
>
> No, I'm saying that your Cleanup class is about as ugly as the try:
> finally:. It just shifts the ugliness around. There is a way to use the
> with statement to make things look better and more readable in certain
> situations, namely where there is some boilerplate that you would
> otherwise repeat in many places using try: finally:. You can encapsulate
> that repetitive code into a class or a @contextmanager generator and
> just call the contextmanager. A generic context manager where you
> register callables doesn't replace any boilerplate. You still repeat all
> of the cleanup code everywhere. What's more, because you have to shove
> everything into a callable, you have significantly less flexibility than
> the try: finally:.

Sorry, but that's meaningless again. You're repeating that my house has no
garbage in it. And you complain that it would be work to add garbage to it. Why
do you want that garbage? I think it's nice without it!


> I will admit that you can put the cleanup code closer to the code that
> needs to get cleaned up, but you pay a price for that.

Yes, that's an additional point, and important. I forgot to mention it. Thanks!


Cheers & hth.,

- Alf

Jerry Hill

3/3/2010 9:02:00 PM

0

On Wed, Mar 3, 2010 at 2:32 PM, Alf P. Steinbach <alfps@start.no> wrote:
> I'm not sure what that shows, except that you haven't grokked this yet.

Maybe you could give us an example of how your code should be used,
and how it differs from the other examples people have given? And
maybe a quick example of why you would not want to clean up after a
failed operation?

I've been trying to follow along, and I don't get it either. I guess
that makes me at least the third person that doesn't understand what
you're trying to get across.

--
Jerry

Robert Kern

3/3/2010 9:03:00 PM

0

On 2010-03-03 13:32 PM, Alf P. Steinbach wrote:
> * Robert Kern:
>> On 2010-03-03 11:18 AM, Alf P. Steinbach wrote:
>>> * Robert Kern:
>>>> On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
>>>>> * Mike Kent:
>>>>>> What's the compelling use case for this vs. a simple try/finally?
>>>>>
>>>>> if you thought about it you would mean a simple "try/else".
>>>>> "finally" is
>>>>> always executed. which is incorrect for cleanup
>>>>
>>>> Eh? Failed execution doesn't require cleanup? The example you gave is
>>>> definitely equivalent to the try: finally: that Mike posted.
>>>
>>> Sorry, that's incorrect: it's not.
>>>
>>> With correct code (mine) cleanup for action A is only performed when
>>> action A succeeds.
>>>
>>> With incorrect code cleanup for action A is performed when A fails.
>>
>> Oh?
>>
>> $ cat cleanup.py
>>
>> class Cleanup:
>> def __init__( self ):
>> self._actions = []
>>
>> def call( self, action ):
>> assert( callable( action ) )
>> self._actions.append( action )
>>
>> def __enter__( self ):
>> return self
>>
>> def __exit__( self, x_type, x_value, x_traceback ):
>> while( len( self._actions ) != 0 ):
>> try:
>> self._actions.pop()()
>> except BaseException as x:
>> raise AssertionError( "Cleanup: exception during cleanup" )
>>
>> def print_(x):
>> print x
>>
>> with Cleanup() as at_cleanup:
>> at_cleanup.call(lambda: print_("Cleanup executed without an exception."))
>>
>> with Cleanup() as at_cleanup:
>
> *Here* is where you should
>
> 1) Perform the action for which cleanup is needed.
>
> 2) Let it fail by raising an exception.
>
>
>> at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
>> raise RuntimeError()
>
> With an exception raised here cleanup should of course be performed.
>
> And just in case you didn't notice: the above is not a test of the
> example I gave.
>
>
>> $ python cleanup.py
>> Cleanup executed without an exception.
>> Cleanup execute with an exception.
>> Traceback (most recent call last):
>> File "cleanup.py", line 28, in <module>
>> raise RuntimeError()
>> RuntimeError
>>
>>>> The actions are always executed in your example,
>>>
>>> Sorry, that's incorrect.
>>
>> Looks like it to me.
>
> I'm sorry, but you're
>
> 1) not testing my example which you're claiming that you're testing, and

Then I would appreciate your writing a complete, runnable example that
demonstrates the feature you are claiming. Because it's apparently not
"ensur[ing] some desired cleanup at the end of a scope, even when the scope is
exited via an exception" that you talked about in your original post.

Your sketch of an example looks like mine:

with Cleanup as at_cleanup:
# blah blah
chdir( somewhere )
at_cleanup.call( lambda: chdir( original_dir ) )
# blah blah

The cleanup function gets registered immediately after the first chdir() and
before the second "blah blah". Even if an exception is raised in the second
"blah blah", then the cleanup function will still run. This would be equivalent
to a try: finally:

# blah blah #1
chdir( somewhere )
try:
# blah blah #2
finally:
chdir( original_dir )

and not a try: else:

# blah blah #1
chdir( somewhere )
try:
# blah blah #2
else:
chdir( original_dir )

Now, I assumed that the behavior with respect to exceptions occurring in the
first "blah blah" weren't what you were talking about because until the chdir(),
there is nothing to clean up.

There is no way that the example you gave translates to a try: else: as you
claimed in your response to Mike Kent.

> 2) not even showing anything about your earlier statements, which were
> just incorrect.
>
> You're instead showing that my code works as it should for the case that
> you're testing, which is a bit unnecessary since I knew that, but thanks
> anyway.

It's the case you seem to be talking about in your original post. You seem to
have changed your mind about what you want to talk about. That's fine. We don't
have to stick with the original topic, but I do ask you to acknowledge that you
originally were talking about a feature that "ensure[s] some desired cleanup at
the end of a scope, even when the scope is exited via an exception."

Do you acknowledge this?

> I'm not sure what that shows, except that you haven't grokked this yet.
>
>
>>>> From your post, the scope guard technique is used "to ensure some
>>>> desired cleanup at the end of a scope, even when the scope is exited
>>>> via an exception." This is precisely what the try: finally: syntax is
>>>> for.
>>>
>>> You'd have to nest it. That's ugly. And more importantly, now two people
>>> in this thread (namely you and Mike) have demonstrated that they do not
>>> grok the try functionality and manage to write incorrect code, even
>>> arguing that it's correct when informed that it's not, so it's a pretty
>>> fragile construct, like goto.
>>
>> Uh-huh.
>
> Yeah. Consider that you're now for the third time failing to grasp the
> concept of cleanup for a successful operation.

Oh, I do. But if I didn't want it to run on an exception, I'd just write the
code without any try:s or with:s at all.

# blah blah #1
chdir( somewhere )
# blah blah #2
chdir( original_dir )

>>>> The with statement allows you to encapsulate repetitive boilerplate
>>>> into context managers, but a general purpose context manager like your
>>>> Cleanup class doesn't take advantage of this.
>>>
>>> I'm sorry but that's pretty meaningless. It's like: "A house allows you
>>> to encapsulate a lot of stinking garbage, but your house doesn't take
>>> advantage of that, it's disgustingly clean". Hello.
>>
>> No, I'm saying that your Cleanup class is about as ugly as the try:
>> finally:. It just shifts the ugliness around. There is a way to use
>> the with statement to make things look better and more readable in
>> certain situations, namely where there is some boilerplate that you
>> would otherwise repeat in many places using try: finally:. You can
>> encapsulate that repetitive code into a class or a @contextmanager
>> generator and just call the contextmanager. A generic context manager
>> where you register callables doesn't replace any boilerplate. You
>> still repeat all of the cleanup code everywhere. What's more, because
>> you have to shove everything into a callable, you have significantly
>> less flexibility than the try: finally:.
>
> Sorry, but that's meaningless again. You're repeating that my house has
> no garbage in it.

No, I'm repeatedly saying that I think your solution stinks. I think it's ugly.
I think it's restrictive. I think it does not improve on the available solutions.

> And you complain that it would be work to add garbage
> to it. Why do you want that garbage? I think it's nice without it!

And you are entitled to that opinion. I am giving you mine.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco