Dangerous behavior of list(generator)
in [Python]
|
Prev: Spawning an interactive interpreter in a running python process?
Next: unable to read the __main__ namespace
From: Gregory Ewing on 16 Dec 2009 18:48 exarkun(a)twistedmatrix.com wrote: > Which is unfortunate, because it's not that hard to get StopIteration > without explicitly raising it yourself and this behavior makes it > difficult to debug such situations. It might not be hard if you set out to do it, but in my experience it's pretty rare to end up getting a StopIteration raised accidentally in an unexpected place. -- Greg
From: Albert van der Horst on 18 Dec 2009 12:58 In article <mailman.1818.1260694428.2873.python-list(a)python.org>, Gabriel Genellina <gagsl-py2(a)yahoo.com.ar> wrote: <SNIP> > >Despite a promise in PEP 289, generator expressions semantics isn't >explained in detail in the language reference. I can't tell if the >difference is intentional, accidental, undocumented behavior, an >implementation accident, a bug, or what... Philosophically speaking ... An important feature that is not documented is a severe defect. (important maps to severe). Before it is documented, there can be no discrepancy between specification and implementation so other defects are formally not present in relation to this situation. >-- >Gabriel Genellina > Groetjes Albert. -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert(a)spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
From: Gregory Ewing on 19 Dec 2009 00:15 Albert van der Horst wrote: > An important feature that is not documented is a severe defect. This isn't something that I would expect to find documented under the heading of generator expressions, because it doesn't have anything to do with them. It's an interaction between the iterator protocol and the list() constructor. Any other iterable that leaked a StopIteration exception would cause the same effect. -- Greg
From: Tom Machinski on 30 Dec 2009 18:18 Thanks for the comment and discussion guys. Bottom line, I'm going to have to remove this pattern from my code: foo = (foo for foo in foos if foo.bar).next() I used to have that a lot in cases where not finding at least one valid foo is an actual fatal error. But using StopIteration to signal a fatal condition becomes a bug when interacting with list() as shown in the original post. It would be nice if there was a builtin for "get the first element in a genexp, or raise an exception (which isn't StopIteration)", sort of like: from itertools import islice def first_or_raise(genexp): L = list(islice(genexp, 1)) if not L: raise RuntimeError('no elements found') return L[0] I also think Jean-Paul's had a good point about how the problems in the list/genexp interaction could be addressed. Thank you, -- Tom
From: Steven D'Aprano on 30 Dec 2009 19:01
On Wed, 30 Dec 2009 15:18:11 -0800, Tom Machinski wrote: > Thanks for the comment and discussion guys. > > Bottom line, I'm going to have to remove this pattern from my code: > > foo = (foo for foo in foos if foo.bar).next() I don't see why. What's wrong with it? Unless you embed it in a call to list, or similar, it will explicitly raise StopIteration as expected. > I used to have that a lot in cases where not finding at least one valid > foo is an actual fatal error. What's wrong with the obvious solution? if not any(foo for foo in foos if foo.bar): raise ValueError('need at least one valid foo') > But using StopIteration to signal a fatal > condition becomes a bug when interacting with list() as shown in the > original post. You shouldn't use StopIteration to signal fatal conditions, because that's not what it is for. It's acceptable to catch it when *directly* calling next, but otherwise you should expect that StopIteration will be caught and suppressed by just about anything. > It would be nice if there was a builtin for "get the first element in a > genexp, or raise an exception (which isn't StopIteration)", Not everything needs to be a built-in. def get_first_or_fail(iterable_or_sequence): it = iter(iterable_or_sequence) try: return it.next() # use next(it) in Python 3 except StopIteration: raise ValueError('empty iterable') This is perfectly usable as a helper function, or it's short enough to be used in-line if you prefer. -- Steven |