Easy questions from a python beginner
in [Python]
|
Prev: Why doesn't python's list append() method return the list itself?
Next: Getting started with python on macintosh snow leopard with mysql - need help
From: wheres pythonmonks on 22 Jul 2010 18:34 Okay -- so I promised that I would try the namespace mangling approach, and here's what I have come up with: Approach #1: Pass in the variables to be swapped as strings. (boring) >>> import sys >>> def swap(n1,n2): .... try: .... raise RuntimeException() .... except: .... e,b,t = sys.exc_info() .... ldict = t.tb_frame.f_back.f_locals .... t = ldict[n1]; .... ldict[n1] = ldict[n2] .... ldict[n2] = t .... >>> x = 'A' >>> y = 'B' >>> id(x) 47946650437696 >>> id(y) 47946649710192 >>> swap('x','y') >>> print id(x) 47946649710192 >>> print id(y) 47946650437696 >>> print x,y B A Approach #2: Allow the user to pass in arbitrary objects, takes the id, infer what the variable in by hashing all possible objects, and then apply the swap operation above. >>> def swap2(o1,o2): .... try: .... raise RuntimeException() .... except: .... e,b,t = sys.exc_info() .... ldict = t.tb_frame.f_back.f_locals .... iddict = dict( (id(v), k ) for k,v in ldict.items() ) .... # print id(o1), id(o2) .... n1 = iddict[id(o1)] .... n2 = iddict[id(o2)] .... t = ldict[n1]; .... ldict[n1] = ldict[n2] .... ldict[n2] = t .... >>> print x,y B A >>> swap2(x,y) >>> print x,y A B >>> Now, I want to make the above codes more "Pythonic" -- is there a way to: 1. Get the function's arguments from the perspective of the caller? def f(x): print "caller's view of x = %s" % callersview(x) Then, f(1+2+3) would yield: caller's view of x = 1 + 2 + 3 2. Is there a better way to loopup by id? I'm not very familiar with sys.exc_info, but creating the id->name hash each time seems like overkill. 3. Is there a reference on all the special variables, like __foo__? 4. Is there any work on deparsing (like Perl's deparse) lambda functions to inline algebra and get a performance gain? Thanks again for your input, W ( from Perl-hacker to Python Programmer ) On Sun, Jul 11, 2010 at 2:37 PM, Stephen Hansen <me+list/python(a)ixokai.io> wrote: > On 7/11/10 10:48 AM, wheres pythonmonks wrote: >> I'm an old Perl-hacker, and am trying to Dive in Python. I have some >> easy issues (Python 2.6) >> which probably can be answered in two seconds: >> >> 1. Why is it that I cannot use print in booleans?? e.g.: >>>>> True and print "It is true!" > > Because print is a statement. Statements have to start lines. If you > want to do this, use a function-- in Python 2.6 either via "from > __future__ import print_function" or writing your own, even if its just > a very thing wrapper around the print statement. > >> 2. How can I write a function, "def swap(x,y):..." so that "x = 3; y >> = 7; swap(x,y);" given x=7,y=3?? >> (I want to use Perl's Ref "\" operator, or C's &). >> (And if I cannot do this [other than creating an Int class], is this >> behavior limited to strings, >> tuples, and numbers) > > You can't do that*. Its not limited to any certain type of objects. You > can't manipulate calling scopes: if you really want to do that sort of > explicit namespace mangling, use dictionaries (or objects, really) as > the namespace to mangle and pass them around. > >> 3. Why might one want to store "strings" as "objects" in numpy >> arrays? (Maybe they wouldn't)? > > I don't use numpy. No idea. > >> 4. Is there a way for me to make some function-definitions explicitly >> module-local? > > In what sense? If you prepend them with an underscore, the function > won't be imported with "from x import *". You can also explicitly > control what is imported in that way with a module-level __all__ attribute. > > Now that won't stop someone from doing "import x" and > "x._your_private_function" but Python doesn't believe in enforicng > restrictions. > >> (Actually related to Q3 below: Is there a way to create an anonymous scope?) > > No. You can create a limited anonymous function with lambda, but note it > takes only an expression-- no statements in it. > >> 5. Is there a way for me to introduce a indention-scoped variables in python? >> See for example: http://evanjones.ca/python-pitfall-scope.html > > No. Python only has three scopes historically; local, global, and > builtin. Then post-2.2(ish, I forget) limited nested scoping -- but only > with nested functions, and you can't (until Python 3) re-bind variables > in outer scopes (though you can modify them if they are mutable objects). > > Python's scoping is very basic (we generally think this is a good thing; > others are never happy with it) and is not fully lexical scoped. > >> 6. Is there a Python Checker that enforces Strunk and White and is >> bad English grammar anti-python? (Only half joking) >> http://www.python.org/dev/peps/pep-0008/ > > Check out pylint and/or pychecker, which do various style-based > checking. If you're asking for something else, I can't pierce your > sarcasm to figure out what. > > -- > > Stephen Hansen > ... Also: Ixokai > ... Mail: me+list/python (AT) ixokai (DOT) io > ... Blog: http://meh.ixokai.io/ > > * Yes, I know its actually possible, to manipulate outer/calling scopes > with frame hacking. This is dangerous / bad / an implementation detail > that one should not rely on or use, generally speaking. If you need to > do this you're writing Java or Perl or C in Python, instead of writing > Python in Python, so are probably doing all kinds of things that are > slow / bad / dangerous / just not taking advantage of Python's strengths. > > > -- > http://mail.python.org/mailman/listinfo/python-list > >
From: Stephen Hansen on 22 Jul 2010 19:59 On 7/22/10 3:34 PM, wheres pythonmonks wrote: > Now, I want to make the above codes more "Pythonic" -- is there a way to: > > 1. Get the function's arguments from the perspective of the caller? > > def f(x): > print "caller's view of x = %s" % callersview(x) > > Then, f(1+2+3) would yield: > caller's view of x = 1 + 2 + 3 No. You can use frame hacking to dig up the locals of the calling frame as you did (such a horrible abuse of the poor language :)), there's no way to determine with any accuracy what exactly they decided to pass into you to get 'x'. I.e., I'm pretty sure its impossible to tell the difference between: x = 3 f(x) And: x = 3 f(3) From within f()'s perspective. This kind of thing gets asked a lot when people don't understand Python's name binding and object passing characteristics and come from other languages. It especially surprises people sometimes to learn that: x = 30000 f(x) f(30000) That in this case, 'f' will receive two completely different objects-- different id's and all. And yet: x = 3 f(x) f(3) In this case, each call will receive the exact same object. > 2. Is there a better way to loopup by id? I'm not very familiar with > sys.exc_info, but creating the id->name hash each time seems like > overkill. Do you mean given a certain 'id', look up what object it is? No there's no way to do that. An id in Python is just informational, and an implementation detail. The language doesn't provide any meaningful tools you can use with them. I mean there's gc.get_objects(), but that doesn't return everything. > 3. Is there a reference on all the special variables, like __foo__? http://docs.python.org/reference/datamodel.html > 4. Is there any work on deparsing (like Perl's deparse) lambda > functions to inline algebra and get a performance gain? There's nothing like that which I'm aware of. You can use projects like numpy or cython to get performance gain in important areas. -- Stephen Hansen ... Also: Ixokai ... Mail: me+list/python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/
From: Carl Banks on 22 Jul 2010 20:12 On Jul 22, 3:34 pm, wheres pythonmonks <wherespythonmo...(a)gmail.com> wrote: > Okay -- so I promised that I would try the namespace mangling > approach, and here's what I have come up with: > > Approach #1: Pass in the variables to be swapped as strings. (boring) > > >>> import sys > >>> def swap(n1,n2): > > ... try: > ... raise RuntimeException() > ... except: > ... e,b,t = sys.exc_info() > ... ldict = t.tb_frame.f_back.f_locals > ... t = ldict[n1]; > ... ldict[n1] = ldict[n2] > ... ldict[n2] = t > ...>>> x = 'A' > >>> y = 'B' > >>> id(x) > 47946650437696 > >>> id(y) > 47946649710192 > >>> swap('x','y') > >>> print id(x) > 47946649710192 > >>> print id(y) > 47946650437696 > >>> print x,y > > B A Have you tried calling this swap inside a function? I bet you haven't. def test(): x = "A" y = "B" swap("x","y") print x,y > Approach #2: Allow the user to pass in arbitrary objects, takes the > id, infer what the variable in by hashing all possible objects, and > then apply the swap operation above. > > >>> def swap2(o1,o2): > > ... try: > ... raise RuntimeException() > ... except: > ... e,b,t = sys.exc_info() > ... ldict = t.tb_frame.f_back.f_locals > ... iddict = dict( (id(v), k ) for k,v in ldict.items() ) > ... # print id(o1), id(o2) > ... n1 = iddict[id(o1)] > ... n2 = iddict[id(o2)] > ... t = ldict[n1]; > ... ldict[n1] = ldict[n2] > ... ldict[n2] = t > ... > > >>> print x,y > B A > >>> swap2(x,y) > >>> print x,y > A B Same question. > Now, I want to make the above codes more "Pythonic" It's simply not possible (let alone Pythonic), in general, to rebind variables in the namespace of the caller. You were able to do it for a very limited circumstance, when the calling namespace was module-level. It doesn't work when the calling namespace is a function. This is true in Python 2 and 3. IMO, even if it could work, the very act of rebinding variables in another namespace is unPythonic. About the only time I've resorted to it is some metaprogramming tasks, and even then I give the functions that do it very explicit names, and I still feel dirty. > -- is there a way to: > > 1. Get the function's arguments from the perspective of the caller? > > def f(x): > print "caller's view of x = %s" % callersview(x) > > Then, f(1+2+3) would yield: > caller's view of x = 1 + 2 + 3 Nope, other than inspecting the caller's frame. > 2. Is there a better way to loopup by id? I'm not very familiar with > sys.exc_info, but creating the id->name hash each time seems like > overkill. Looking up by id is a bad idea in general. Objects associated with an id can be destroyed, and id be reused. So if you're storing an id, by the time you get to it it could be a different object, or an object that no longer exists. > 3. Is there a reference on all the special variables, like __foo__? Python Language Reference > 4. Is there any work on deparsing (like Perl's deparse) lambda > functions to inline algebra and get a performance gain? psyco (q.g.) might help, not sure if it'll help much for lambdas, though. > Thanks again for your input, > > W > > ( from Perl-hacker to Python Programmer ) No offense, but you seem like you're still tying to be a hacker. If that's what you want, fine, but generally speaking (and particularly for Python), you are going to have a better experience if you do it the language's way. And just to throw this out there, based on your questions I think it's possible that Ruby would fit your style better. (It lets you play fast and loose with namespaces and code blocks and such.) Carl Banks
From: wheres pythonmonks on 22 Jul 2010 22:47 Thanks for pointing out that swap (and my swap2) don't work everywhere -- is there a way to get it to work inside functions? "No offense, but you seem like you're still tying to be a hacker. If that's what you want, fine, but generally speaking (and particularly for Python), you are going to have a better experience if you do it the language's way." None taken, but I always think that it is the language's job to express my thoughts... I don't like to think that my thoughts are somehow constrained by the language. The truth is that I don't intend to use these approaches in anything serious. However, I've been known to do some metaprogramming from time to time. In a recent application, I pass in a list of callables (lambdas) to be evaluated repeatedly. Clearly, a superior solution is to pass a single lambda that returns a list. [Less function call dispatches] However, it might be more efficient to avoid the function call overhead completely and pass-in a string which is substituted into a string code block, compiled, and executed. W On Thu, Jul 22, 2010 at 8:12 PM, Carl Banks <pavlovevidence(a)gmail.com> wrote: > On Jul 22, 3:34 pm, wheres pythonmonks <wherespythonmo...(a)gmail.com> > wrote: >> Okay -- so I promised that I would try the namespace mangling >> approach, and here's what I have come up with: >> >> Approach #1: Pass in the variables to be swapped as strings. (boring) >> >> >>> import sys >> >>> def swap(n1,n2): >> >> ... try: >> ... raise RuntimeException() >> ... except: >> ... e,b,t = sys.exc_info() >> ... ldict = t.tb_frame.f_back.f_locals >> ... t = ldict[n1]; >> ... ldict[n1] = ldict[n2] >> ... ldict[n2] = t >> ...>>> x = 'A' >> >>> y = 'B' >> >>> id(x) >> 47946650437696 >> >>> id(y) >> 47946649710192 >> >>> swap('x','y') >> >>> print id(x) >> 47946649710192 >> >>> print id(y) >> 47946650437696 >> >>> print x,y >> >> B A > > Have you tried calling this swap inside a function? I bet you > haven't. > > def test(): > x = "A" > y = "B" > swap("x","y") > print x,y > > >> Approach #2: Allow the user to pass in arbitrary objects, takes the >> id, infer what the variable in by hashing all possible objects, and >> then apply the swap operation above. >> >> >>> def swap2(o1,o2): >> >> ... try: >> ... raise RuntimeException() >> ... except: >> ... e,b,t = sys.exc_info() >> ... ldict = t.tb_frame.f_back.f_locals >> ... iddict = dict( (id(v), k ) for k,v in ldict.items() ) >> ... # print id(o1), id(o2) >> ... n1 = iddict[id(o1)] >> ... n2 = iddict[id(o2)] >> ... t = ldict[n1]; >> ... ldict[n1] = ldict[n2] >> ... ldict[n2] = t >> ... >> >> >>> print x,y >> B A >> >>> swap2(x,y) >> >>> print x,y >> A B > > Same question. > > >> Now, I want to make the above codes more "Pythonic" > > It's simply not possible (let alone Pythonic), in general, to rebind > variables in the namespace of the caller. > > You were able to do it for a very limited circumstance, when the > calling namespace was module-level. It doesn't work when the calling > namespace is a function. This is true in Python 2 and 3. > > IMO, even if it could work, the very act of rebinding variables in > another namespace is unPythonic. About the only time I've resorted to > it is some metaprogramming tasks, and even then I give the functions > that do it very explicit names, and I still feel dirty. > > >> -- is there a way to: >> >> 1. Get the function's arguments from the perspective of the caller? >> >> def f(x): >> print "caller's view of x = %s" % callersview(x) >> >> Then, f(1+2+3) would yield: >> caller's view of x = 1 + 2 + 3 > > Nope, other than inspecting the caller's frame. > >> 2. Is there a better way to loopup by id? I'm not very familiar with >> sys.exc_info, but creating the id->name hash each time seems like >> overkill. > > Looking up by id is a bad idea in general. Objects associated with an > id can be destroyed, and id be reused. So if you're storing an id, by > the time you get to it it could be a different object, or an object > that no longer exists. > > >> 3. Is there a reference on all the special variables, like __foo__? > > Python Language Reference > > >> 4. Is there any work on deparsing (like Perl's deparse) lambda >> functions to inline algebra and get a performance gain? > > psyco (q.g.) might help, not sure if it'll help much for lambdas, > though. > >> Thanks again for your input, >> >> W >> >> ( from Perl-hacker to Python Programmer ) > > No offense, but you seem like you're still tying to be a hacker. If > that's what you want, fine, but generally speaking (and particularly > for Python), you are going to have a better experience if you do it > the language's way. > > And just to throw this out there, based on your questions I think it's > possible that Ruby would fit your style better. (It lets you play > fast and loose with namespaces and code blocks and such.) > > > Carl Banks > -- > http://mail.python.org/mailman/listinfo/python-list >
From: Steven D'Aprano on 22 Jul 2010 23:45
On Thu, 22 Jul 2010 18:34:11 -0400, wheres pythonmonks wrote: > Okay -- so I promised that I would try the namespace mangling approach, > and here's what I have come up with: > > Approach #1: Pass in the variables to be swapped as strings. (boring) Boring and slow and stupid. It makes an interesting trick to prove it can (almost) be done, but for production use? No way. There is nothing that *needs* call-by-reference that can't be done just as effectively using a slightly different approach. This approach also has the fatal flaw that not all objects have names, or have only one name. In Python, objects are often anonymous, and the standard Python idiom for swapping works fine for them: a = [1, 2, 3, "d", 5] b = ["a", "b", "c", 4, "e"] a[3], b[3] = b[3], a[3] I expect your hack to choke and die if you try swapping anonymous objects. [snip successful test of swapping using named globals] As Carl already pointed out, this doesn't work outside of the global scope. This demonstrates an interesting lesson re testing. It's not enough to test these hacks with names in the global scope. You also need to test them with names in a local and nested scope, and in classes. [...] > Now, I want to make the above codes more "Pythonic" The way to make them more Pythonic is to avoid them like the plague. One of the differences in philosophy between Perl and Python is that Python doesn't approve of doing things just because you can. There is very little interest in or respect given to hacking Python. We're programmers, not hackers, and we look for the most simple way to do things, not the gnarliest or trickiest or least obvious way. (Any yet, as a general rule, neither does Python particular get in your way. We have a saying: we're all adults here, if you want to shoot yourself in the foot, go right ahead.) Occasionally, *very* occasionally, that makes it hard to do something that would be simple in another language. But much rarer than you might think, and a lot of popular programming idioms are work-arounds that are simply not needed in Python. E.g. call-by-reference itself was a work- around to prevent unnecessary copying of large data structures like arrays in languages like Algol and Pascal (or so I understand). When Java guys come here and start asking how to implement design patterns in Python, 9 times out of 10 the answer is "you don't need it" rather than "you can't do it". It's not quite true that we consider "hack" to be a pejorative, we're certainly true of the positive uses of the word. But as a general rule the Python community looks rather askance at anything that might be described as a hack. We're far more likely to describe something as a dirty hack than a cool, nifty or amazing hack. It's not true, as some folks say, that there is "only one way to do it" in Python. But the Python ideal is for there to be "one obvious way to do it". If there's an obvious way, why would you look for an obscure, tricky hack? You better have a good reason for not using the obvious way, because writing the code is the *smallest* part of the job. You need to test it, and debug it, and maintain it, and they're all much, much harder. > is there a way to: > > 1. Get the function's arguments from the perspective of the caller? I don't understand the question. If the caller passes arg = [1,2,3], then both the caller and the called function agree that the argument is the list [1,2,3]. Both caller and callee see the same list. It would be a pretty strange programming language where the called function saw something different from what the caller passed... > 2. Is there a better way to loopup by id? I'm not very familiar with > sys.exc_info, but creating the id->name hash each time seems like > overkill. No. > 3. Is there a reference on all the special variables, like __foo__? If you're talking about "magic methods" and special attributes, like obj.__len__, then you should read this: http://docs.python.org/reference/datamodel.html This may also be helpful: http://www.ironpythoninaction.com/magic-methods.html If you're talking about special globals like __name__, I don't think there are any others. __name__ is (confusingly) documented here: http://docs.python.org/library/__main__.html > 4. Is there any work on deparsing (like Perl's deparse) lambda > functions to inline algebra and get a performance gain? No. How would you do that? If I have something like this: def my_func(arg): return some_class(1, 2, 3, callback=lambda x: arg.method(x)) how would you inline that? *Where* would you inline that? Perhaps you're thinking that lambdas are commonly used like this: def my_func(args): f = lambda a, b, c: 3*a + 4*b -2*c results = [] for x in args: results.append(f(2, x, 3)) results.append(f(x, x**2, x**3) return results then, yes, that lambda could be inlined. But that's a rare and uncommon use for lambdas, and some would say unpythonic. Even if you could inline it, CPython isn't big on micro-optimizations. It's generally considered that the extra complexity and risk of bugs isn't worth the tiny performance gain. Simplicity itself is a virtue, and many optimizations tend to be anything but simple, if not dirty hacks. -- Steven |