A "scopeguard" for Python
in [Python]
|
From: Robert Kern on 4 Mar 2010 12:00 On 2010-03-04 10:32 AM, Jean-Michel Pichavant wrote: > Alf P. Steinbach wrote: >> * Jean-Michel Pichavant: >>> Alf P. Steinbach wrote: >>>>> From your post, the scope guard technique is used "to ensure some >>>>> desired cleanup at the end of a scope, even when the scope is >>>>> exited via an exception." This is precisely what the try: finally: >>>>> syntax is for. >>>> >>>> You'd have to nest it. That's ugly. And more importantly, now two >>>> people in this thread (namely you and Mike) have demonstrated that >>>> they do not grok the try functionality and manage to write incorrect >>>> code, even arguing that it's correct when informed that it's not, so >>>> it's a pretty fragile construct, like goto. >>> >>> You want to execute some cleanup when things go wrong, use try >>> except. You want to do it when things go right, use try else. You >>> want to cleanup no matter what happen, use try finally. >>> >>> There is no need of any Cleanup class, except for some technical >>> alternative concern. >> >> Have you considered that your argument applies to the "with" construct? >> >> You have probably not realized that. >> >> But let me force it on you: when would you use "with"? >> >> Check if that case is covered by your argument above. >> >> Now that you've been told about the "with" angle, don't you think it's >> a kind of weakness in your argument that it calls for removing "with" >> from the language? >> >> I recommend that you think about why your argument is invalid. >> >> Or, as I like to say, why your argument is completely bogus. >> >> >> Cheers & hth., >> >> - Alf > I am using python 2.5, so I know nothing about the with statement, You can try it out using "from __future__ import with_statement". > and > it may possible my arguments apply to it, you could remove it from the > language, it wouldn't bother me at all. > I just don't see in what you've written (adding a class, with some > __entry__, __exit__ protocol, using a with statement) what cannot be > achieved with a try statement in its simpliest form. > > Try except may be lame and noobish, but it works, is easy to read and > understood at first glance. > It looks like to me that 'with' statements are like decorators: > overrated. Sometimes people could write simple readable code, but yet > they're tempted by the geek side of programming: using complex > constructs when there's no need to. I myself cannot resist sometimes ;-) PEP 343 is a good introduction to the real uses of the with: statement. http://www.python.org/dev/peps/pep-0343/ Basically, it allows you to package up your initialization and cleanup code into objects, stick them in your library, unit test them thoroughly, etc. so you don't have to repeat them everywhere and possibly get them wrong. It's DRY in action. Where Alf's Cleanup class goes wrong, in my opinion, is that it does not package up any code to avoid repetition. You still repeat the same cleanup code everywhere you use it, so it is no better than try: finally:. It is not a real use case of the with: statement. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
From: Alf P. Steinbach on 4 Mar 2010 12:02 * Robert Kern: > On 2010-03-04 09:48 AM, Alf P. Steinbach wrote: >> * Jean-Michel Pichavant: >>> Alf P. Steinbach wrote: >>>>> From your post, the scope guard technique is used "to ensure some >>>>> desired cleanup at the end of a scope, even when the scope is exited >>>>> via an exception." This is precisely what the try: finally: syntax >>>>> is for. >>>> >>>> You'd have to nest it. That's ugly. And more importantly, now two >>>> people in this thread (namely you and Mike) have demonstrated that >>>> they do not grok the try functionality and manage to write incorrect >>>> code, even arguing that it's correct when informed that it's not, so >>>> it's a pretty fragile construct, like goto. >>> >>> You want to execute some cleanup when things go wrong, use try except. >>> You want to do it when things go right, use try else. You want to >>> cleanup no matter what happen, use try finally. >>> >>> There is no need of any Cleanup class, except for some technical >>> alternative concern. >> >> Have you considered that your argument applies to the "with" construct? >> >> You have probably not realized that. >> >> But let me force it on you: when would you use "with"? > > When there is a specific context manager that removes the need for > boilerplate. That's "cleanup no matter what happen". >> Check if that case is covered by your argument above. >> >> Now that you've been told about the "with" angle, don't you think it's a >> kind of weakness in your argument that it calls for removing "with" from >> the language? > > No, it only argues that "with Cleanup():" is supernumerary. I don't know what "supernumerary" means, but to the degree that the argument says anything about a construct that is not 'finally', it says the same about general "with". So whatever you mean by supernumerary, you're saying that the argument implies that "with" is supernumerary. This is starting to look like some earlier discussions in this group, where even basic logic is denied. Cheers, - Alf
From: Robert Kern on 4 Mar 2010 12:20 On 2010-03-04 10:56 AM, Alf P. Steinbach wrote: > * Robert Kern: >> On 2010-03-03 18:49 PM, Alf P. Steinbach wrote: >>> * Robert Kern: > [snip] >>>> can you >>>> understand why we might think that you were saying that try: finally: >>>> was wrong and that you were proposing that your code was equivalent to >>>> some try: except: else: suite? >>> >>> No, not really. His code didn't match the semantics. Changing 'finally' >>> to 'else' could make it equivalent. >> >> Okay, please show me what you mean by "changing 'finally' to 'else'." >> I think you are being hinty again. It's not helpful. > [snip middle of this paragraph] >> Why do you think that we would interpret those words to mean that you >> wanted the example you give just above? > > There's an apparent discrepancy between your call for an example and > your subsequent (in the same paragraph) reference to the example given. > > But as to why I assumed that that example, or a similar correct one, > would be implied, it's the only meaningful interpretation. > > Adopting a meaningless interpretation when a meaningful exists is > generally just adversarial, but in this case I was, as you pointed out, > extremely unclear, and I'm sorry: I should have given such example up > front. Will try to do so. Thank you. I appreciate it. > [snip] >> >>>> There are a couple of ways to do this kind of cleanup depending on the >>>> situation. Basically, you have several different code blocks: >>>> >>>> # 1. Record original state. >>>> # 2. Modify state. >>>> # 3. Do stuff requiring the modified state. >>>> # 4. Revert to the original state. >>>> >>>> Depending on where errors are expected to occur, and how the state >>>> needs to get modified and restored, there are different ways of >>>> arranging these blocks. The one Mike showed: >>>> >>>> # 1. Record original state. >>>> try: >>>> # 2. Modify state. >>>> # 3. Do stuff requiring the modified state. >>>> finally: >>>> # 4. Revert to the original state. >>>> >>>> And the one you prefer: >>>> >>>> # 1. Record original state. >>>> # 2. Modify state. >>>> try: >>>> # 3. Do stuff requiring the modified state. >>>> finally: >>>> # 4. Revert to the original state. >>>> >>>> These differ in what happens when an error occurs in block #2, the >>>> modification of the state. In Mike's, the cleanup code runs; in yours, >>>> it doesn't. For chdir(), it really doesn't matter. Reverting to the >>>> original state is harmless whether the original chdir() succeeds or >>>> fails, and chdir() is essentially atomic so if it raises an exception, >>>> the state did not change and nothing needs to be cleaned up. >>>> >>>> However, not all block #2s are atomic. Some are going to fail partway >>>> through and need to be cleaned up even though they raised an >>>> exception. Fortunately, cleanup can frequently be written to not care >>>> whether the whole thing finished or not. >>> >>> Yeah, and there are some systematic ways to handle these things. You >>> might look up Dave Abraham's levels of exception safety. Mostly his >>> approach boils down to making operations effectively atomic so as to >>> reduce the complexity: ideally, if an operation raises an exception, >>> then it has undone any side effects. >>> >>> Of course it can't undo the launching of an ICBM, for example... >>> >>> But ideally, if it could, then it should. >> >> I agree. Atomic operations like chdir() help a lot. But this is >> Python, and exceptions can happen in many different places. If you're >> not just calling an extension module function that makes a >> known-atomic system call, you run the risk of not having an atomic >> operation. >> >>> If you call the possibly failing operation "A", then that systematic >>> approach goes like this: if A fails, then it has cleaned up its own >>> mess, but if A succeeds, then it's the responsibility of the calling >>> code to clean up if the higher level (multiple statements) operation >>> that A is embedded in, fails. >>> >>> And that's what Marginean's original C++ ScopeGuard was designed for, >>> and what the corresponding Python Cleanup class is designed for. >> >> And try: finally:, for that matter. > > Not to mention "with". > > Some other poster made the same error recently in this thread; it is a > common fallacy in discussions about programming, to assume that since > the same can be expressed using lower level constructs, those are all > that are required. > > If adopted as true it ultimately means the removal of all control > structures above the level of "if" and "goto" (except Python doesn't > have "goto"). What I'm trying to explain is that the with: statement has a use even if Cleanup doesn't. Arguing that Cleanup doesn't improve on try: finally: does not mean that the with: statement doesn't improve on try: finally:. >>>> Both formulations can be correct (and both work perfectly fine with >>>> the chdir() example being used). Sometimes one is better than the >>>> other, and sometimes not. You can achieve both ways with either your >>>> Cleanup class or with try: finally:. >>>> >>>> I am still of the opinion that Cleanup is not an improvement over try: >>>> finally: and has the significant ugliness of forcing cleanup code into >>>> callables. This significantly limits what you can do in your cleanup >>>> code. >>> >>> Uhm, not really. :-) As I see it. >> >> Well, not being able to affect the namespace is a significant >> limitation. Sometimes you need to delete objects from the namespace in >> order to ensure that their refcounts go to zero and their cleanup code >> gets executed. > > Just a nit (I agree that a lambda can't do this, but as to what's > required): assigning None is sufficient for that[1]. Yes, but no callable is going to allow you to assign None to names in that namespace, either. Not without sys._getframe() hackery, in any case. > However, note that the current language doesn't guarantee such cleanup, > at least as far as I know. > > So while it's good practice to support it, to do everything to let it > happen, it's presumably bad practice to rely on it happening. > > >> Tracebacks will keep the namespace alive and all objects in it. > > Thanks!, I hadn't thought of connecting that to general cleanup actions. > > It limits the use of general "with" in the same way. Not really. It's easy to write context managers that do that. You put the initialization code in the __enter__() method, assign whatever objects you want to keep around through the with: clause as attributes on the manager, then delete those attributes in the __exit__(). Or, you use the @contextmanager decorator to turn a generator into a context manager, and you just assign to local variables and del them in the finally: clause. What you can't do is write a generic context manager where the initialization happens inside the with: clause and the cleanup actions are registered callables. That does not allow you to affect the namespace. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
From: Robert Kern on 4 Mar 2010 12:30 On 2010-03-04 11:02 AM, Alf P. Steinbach wrote: > * Robert Kern: >> On 2010-03-04 09:48 AM, Alf P. Steinbach wrote: >>> * Jean-Michel Pichavant: >>>> Alf P. Steinbach wrote: >>>>>> From your post, the scope guard technique is used "to ensure some >>>>>> desired cleanup at the end of a scope, even when the scope is exited >>>>>> via an exception." This is precisely what the try: finally: syntax >>>>>> is for. >>>>> >>>>> You'd have to nest it. That's ugly. And more importantly, now two >>>>> people in this thread (namely you and Mike) have demonstrated that >>>>> they do not grok the try functionality and manage to write incorrect >>>>> code, even arguing that it's correct when informed that it's not, so >>>>> it's a pretty fragile construct, like goto. >>>> >>>> You want to execute some cleanup when things go wrong, use try except. >>>> You want to do it when things go right, use try else. You want to >>>> cleanup no matter what happen, use try finally. >>>> >>>> There is no need of any Cleanup class, except for some technical >>>> alternative concern. >>> >>> Have you considered that your argument applies to the "with" construct? >>> >>> You have probably not realized that. >>> >>> But let me force it on you: when would you use "with"? >> >> When there is a specific context manager that removes the need for >> boilerplate. > > That's "cleanup no matter what happen". For the "# Do stuff" block, yes. For the initialization block, you can write a context manager to do it either way, as necessary. >>> Check if that case is covered by your argument above. >>> >>> Now that you've been told about the "with" angle, don't you think it's a >>> kind of weakness in your argument that it calls for removing "with" from >>> the language? >> >> No, it only argues that "with Cleanup():" is supernumerary. > > I don't know what "supernumerary" means, http://www.merriam-webster.com/dictionary/supernumerary > but to the degree that the > argument says anything about a construct that is not 'finally', it says > the same about general "with". He's ignorant of the use cases of the with: statement, true. Given only your example of the with: statement, it is hard to fault him for thinking that try: finally: wouldn't suffice. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
From: Michael Rudolf on 4 Mar 2010 12:50
Am 04.03.2010 18:20, schrieb Robert Kern: > > What I'm trying to explain is that the with: statement has a use even if > Cleanup doesn't. Arguing that Cleanup doesn't improve on try: finally: > does not mean that the with: statement doesn't improve on try: finally:. Yes, the with-statement rocks :) I suggested such a thing a few days ago in another thread, where OP wanted a "silent" keyword like: silent: do_stuff() would be equivalent to: try: do_stuff() except: pass Of course catching *all* exceptions was a bad idea, so I came up with the code below and now I actually like it and use it myself :) ---------snip--------- To your first question about a "silenced" keyword: you could emulate this with context managers I guess. Something like (untested, just a quick mockup how it could look): class silenced: def __init__(self, *silenced): self.exceptions=tuple(silenced) #just to be explicit def __enter__(self): return self #dito def __exit__(self, type, value, traceback): for ex in self.exceptions: if isinstance(value, ex): return True #supresses exception So: with silenced(os.Error): os.remove(somefile) Would translate to: try: os.remove(somefile) except os.Error: pass One nice thing about this approach would be that you can alias a set of exceptions with this: idontcareabouttheseerrors=silenced(TypeError, ValueError, PEBCAKError, SyntaxError, EndOfWorldError, 1D10T_Error) with idontcareabouttheseerrors: do_stuff() Regards, Michael |