How to guard against bugs like this one?
in [Python]
|
From: Carl Banks on 1 Feb 2010 23:01 On Feb 1, 6:34 pm, kj <no.em...(a)please.post> wrote: > Both scripts live in a directory filled with *hundreds* little > one-off scripts like the two of them. I'll call this directory > myscripts in what follows. [snip] > How can the average Python programmer guard against this sort of > time-devouring bug in the future (while remaining a Python programmer)? Don't put hundreds of little one-off scripts in single directory. Python can't save you from polluting your own namespace. Don't choose such generic names for modules. Keep in mind module names are potentially globally visible and any sane advice you ever heard about globals is to use descriptive names. I instinctively use adjectives, compound words, and abstract nouns for the names of all my modules so as to be more descriptive, and to avoid name conflicts with classes and variables. Also learn to debug better. Carl Banks
From: Carl Banks on 1 Feb 2010 23:12 On Feb 1, 7:33 pm, Tim Chase <python.l...(a)tim.thechases.com> wrote: > Stephen Hansen wrote: > > First, I don't shadow built in modules. Its really not very hard to avoid. > > Given the comprehensive nature of the batteries-included in > Python, it's not as hard to accidentally shadow a built-in, > unknown to you, but yet that is imported by a module you are > using. The classic that's stung me enough times (and many others > on c.l.p and other forums, as a quick google evidences) such that > I *finally* remember: > > bash$ touch email.py > bash$ python > ... > >>> import smtplib > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python2.5/smtplib.py", line 46, in <module> > import email.Utils > ImportError: No module named Utils > > Using "email.py" is an innocuous name for a script/module you > might want to do emailish things, and it's likely you'll use > smtplib in the same code...and kablooie, things blow up even if > your code doesn't reference or directly use the built-in email.py. email.py is not an innocuous name, it's a generic name in a global namespace, which is a Bad Thing. Plus what does a script or module called "email.py" actually do? Send email? Parse email? "email" is terrible name for a module and you deserve what you got for using it. Name your modules "send_email.py" or "sort_email.py" or if it's a library module of related functions, "email_handling.py". Modules and scripts do things (usually), they should be given action words as names. (**) Questionable though it be, if the Standard Library wants to use an "innocuous" name, It can. Carl Banks
From: Mark Dickinson on 2 Feb 2010 04:20 On Feb 2, 3:28 am, Steven D'Aprano <ste...(a)REMOVE.THIS.cybersource.com.au> wrote: > > There is no module numbers in the standard library, at least not in 2.5. It's new in 2.6 (and 3.0, I think; it's there in 3.1, anyway). It provides abstract base classes for numeric types; see the fractions module source for some of the ways it can be used. Here are the docs: http://docs.python.org/library/numbers.html and also PEP 3141, on which it's based: http://www.python.org/dev/peps/pep-3141/ -- Mark
From: Jean-Michel Pichavant on 2 Feb 2010 05:49 Carl Banks wrote: > On Feb 1, 7:33 pm, Tim Chase <python.l...(a)tim.thechases.com> wrote: > >> Stephen Hansen wrote: >> >>> First, I don't shadow built in modules. Its really not very hard to avoid. >>> >> Given the comprehensive nature of the batteries-included in >> Python, it's not as hard to accidentally shadow a built-in, >> unknown to you, but yet that is imported by a module you are >> using. The classic that's stung me enough times (and many others >> on c.l.p and other forums, as a quick google evidences) such that >> I *finally* remember: >> >> bash$ touch email.py >> bash$ python >> ... >> >>> import smtplib >> Traceback (most recent call last): >> File "<stdin>", line 1, in <module> >> File "/usr/lib/python2.5/smtplib.py", line 46, in <module> >> import email.Utils >> ImportError: No module named Utils >> >> Using "email.py" is an innocuous name for a script/module you >> might want to do emailish things, and it's likely you'll use >> smtplib in the same code...and kablooie, things blow up even if >> your code doesn't reference or directly use the built-in email.py. >> > > > email.py is not an innocuous name, it's a generic name in a global > namespace, which is a Bad Thing. Plus what does a script or module > called "email.py" actually do? Send email? Parse email? "email" is > terrible name for a module and you deserve what you got for using it. > > Name your modules "send_email.py" or "sort_email.py" or if it's a > library module of related functions, "email_handling.py". Modules and > scripts do things (usually), they should be given action words as > names. > > > (**) Questionable though it be, if the Standard Library wants to use > an "innocuous" name, It can. > > > Carl Banks > That does not solve anything, if the smtplib follows your advice, then you'll be shadowing its send_email module. The only way to avoid collision would be to name your module __PDSFLSDF_send_email__13221sdfsdf__.py That way, the probabilty you'd shadow one package hidden module is below the probability that Misses Hilton ever says something relevant. However nobody wants to use such names. Stephen gave good advices in this thread that helps avoiding this issue. JM
From: kj on 2 Feb 2010 09:13
Let me preface everything by thanking you and all those who replied for their comments. I have only one follow-up question (or rather, set of related questions) that I'm very keen about, plus a bit of a vent at the end. In <pan.2010.02.02.03.28.54(a)REMOVE.THIS.cybersource.com.au> Steven D'Aprano <steven(a)REMOVE.THIS.cybersource.com.au> writes: >As for fixing it, unfortunately it's not quite so simple to fix without >breaking backwards-compatibility. The opportunity to do so for Python 3.0 >was missed. This last point is to me the most befuddling of all. Does anyone know why this opportunity was missed for 3.0? Anyone out there with the inside scoop on this? Was the fixing of this problem discussed in some PEP or some mailing list thread? (I've tried Googling this but did not hit on the right keywords to bring up the deliberations I'm looking for.) ~k [NB: as I said before, what follows begins to slide into a vent, and is quite unimportant; I've left it, for whatever grain of truth it may contain, as an grossly overgrown PS; feel free to ignore it, I'm *by far* most interested in the question stated in the paragraph right above, because it will give me, I hope, a better sense of where the biggest obstacles to fixing this problem lie.] P.S. Yes, I see the backwards-compatibility problem, but that's what rolling out a whole new versions is good for; it's a bit of a fresh start. I remember hearing GvR's Google Talk on the coming Python 3, which was still in the works then, and being struck by the sheer *modesty* of the proposed changes (while the developers of the mythical Perl6 seemed to be on a quest for transcendence to a Higher Plane of Programming, as they still are). In particular the business with print -> print() seemed truly bizarre to me: this is a change that will break a *huge* volume of code, and yet, judging by the rationale given for it, the change solves what are, IMHO, a relatively minor annoyances. Python's old print statement is, I think, at most a tiny little zit invisible to all but those obsessed with absolute perfection. And I can't imagine that whatever would be required to fix Python's import system could break more code than redefining the rules for a workhorse like print. In contrast, the Python import problem is a ticking bomb potentially affecting all code that imports other modules. All that needs to happen is that, in a future release of Python, some new standard module emerges (like numbers.py emerged in 2.6), and this module is imported by some module your code imports. Boom! Note that it was only coincidental that the bug I reported in this thread occurred in a script I wrote recently. I could have written both scripts before 2.6 was released, and the new numbers.py along with it; barring the uncanny clairvoyance of some responders, there would have been, at the time, absolutely no plausible reason for not naming one of the two scripts numbers.py. To the argument that the import system can't be easily fixed because it breaks existing code, one can reply that the *current* import system already breaks existing code, as illustrated by the example I've given in this thread: this could have easily been old pre-2.6 code that got broken just because Python decided to add numbers.py to the distribution. (Yes, Python can't guarantee that the names of new standard modules won't clash with the names of existing local modules, but this is true for Perl as well, and due to Perl's module import scheme (and naming conventions), a scenario like the one I presented in this thread would have been astronomically improbable. The Perl example shows that the design of the module import scheme and naming conventions for standard modules can go a long way to minimize the consequences of this unavoidable potential for future name clashes.) |