Is python not good enough?
in [Python]
|
From: hackingKK on 16 Jan 2010 02:02 On Saturday 16 January 2010 08:01 AM, Nobody wrote: > On Fri, 15 Jan 2010 12:34:17 -0800, John Nagle wrote: > > >> Actually, no. It's quite possible to make a Python implementation that >> runs fast. It's just that CPython, a naive interpreter, is too primitive >> to do it. I was really hoping that Google would put somebody good at >> compilers in charge of Python and bring it up to production speed. >> "production"? >> >> Look at Shed Skin, a hard-code compiler for Python >> > A hard-code compiler for the subset of Python which can easily be compiled. > > Shed Skin has so many restrictions that it isn't really accurate to > describe the language which it supports as "Python". > +1 > > Hardly any real-world Python code can be compiled with Shed Skin. Some of > it could be changed without too much effort, although most of that is the > kind of code which wouldn't look any different if it was implemented in > C++ or Java. > Happy hacking. Krishnakant.
From: John Nagle on 16 Jan 2010 14:17 Nobody wrote: > On Fri, 15 Jan 2010 12:34:17 -0800, John Nagle wrote: > >> Actually, no. It's quite possible to make a Python implementation that >> runs fast. It's just that CPython, a naive interpreter, is too primitive >> to do it. I was really hoping that Google would put somebody good at >> compilers in charge of Python and bring it up to production speed. >> >> Look at Shed Skin, a hard-code compiler for Python > > A hard-code compiler for the subset of Python which can easily be compiled. > > Shed Skin has so many restrictions that it isn't really accurate to > describe the language which it supports as "Python". > > Hardly any real-world Python code can be compiled with Shed Skin. Some of > it could be changed without too much effort, although most of that is the > kind of code which wouldn't look any different if it was implemented in > C++ or Java. > > The monomorphism restriction is likely to be particularly onerous: the > type of a variable must be known at compile time; instances of subclasses > are allowed, but you can only call methods which are defined in the > compile-time class. > > If you're writing code which makes extensive use of Python's dynamicity, > making it work with Shed Skin would require as much effort as re-writing > it in e.g. Java, and would largely defeat the point of using Python in the > first place. > > http://shedskin.googlecode.com/files/shedskin-tutorial-0.3.html > > If you want a language to have comparable performance to C++ or Java, you > have to allow some things to be fixed at compile-time. There's a reason > why C++ and Java support both virtual and non-virtual ("final") methods. My point is that Python is a good language held back by a bad implementation. Python has gotten further with a declaration-free syntax than any other language. BASIC and JavaScript started out declaration-free, and declarations had to be retrofitted. Python has survived without them. (Yes, there are hokey extensions like Psyco declarations and "decorators", but both are marginal concepts.) The key to hard-compiling Python is that you have to compile the whole program, not individual modules. You can't tell how an individual module will be used until you've seen its callers. If the compiler looks at the whole program at once, type inference has a good chance of disambiguating most type issues. If you can see the whole program at once, most dynamism can be detected. What's really needed is to detect the most common case, where objects don't have unexpected dynamism and can be implemented as hard structures. John Nagle
From: Steve Holden on 16 Jan 2010 18:03 John Nagle wrote: > Nobody wrote: >> On Fri, 15 Jan 2010 12:34:17 -0800, John Nagle wrote: >> >>> Actually, no. It's quite possible to make a Python >>> implementation that >>> runs fast. It's just that CPython, a naive interpreter, is too >>> primitive >>> to do it. I was really hoping that Google would put somebody good at >>> compilers in charge of Python and bring it up to production speed. >>> >>> Look at Shed Skin, a hard-code compiler for Python >> >> A hard-code compiler for the subset of Python which can easily be >> compiled. >> >> Shed Skin has so many restrictions that it isn't really accurate to >> describe the language which it supports as "Python". >> >> Hardly any real-world Python code can be compiled with Shed Skin. Some of >> it could be changed without too much effort, although most of that is the >> kind of code which wouldn't look any different if it was implemented in >> C++ or Java. >> >> The monomorphism restriction is likely to be particularly onerous: the >> type of a variable must be known at compile time; instances of subclasses >> are allowed, but you can only call methods which are defined in the >> compile-time class. >> >> If you're writing code which makes extensive use of Python's dynamicity, >> making it work with Shed Skin would require as much effort as re-writing >> it in e.g. Java, and would largely defeat the point of using Python in >> the >> first place. >> >> http://shedskin.googlecode.com/files/shedskin-tutorial-0.3.html >> >> If you want a language to have comparable performance to C++ or Java, you >> have to allow some things to be fixed at compile-time. There's a reason >> why C++ and Java support both virtual and non-virtual ("final") methods. > > My point is that Python is a good language held back by a bad > implementation. Python has gotten further with a declaration-free syntax > than any other language. BASIC and JavaScript started out > declaration-free, > and declarations had to be retrofitted. Python has survived without them. > (Yes, there are hokey extensions like Psyco declarations and "decorators", > but both are marginal concepts.) > > The key to hard-compiling Python is that you have to compile the > whole program, not individual modules. You can't tell how an individual > module will be used until you've seen its callers. If the compiler > looks at the whole program at once, type inference has a good chance of > disambiguating most type issues. > > If you can see the whole program at once, most dynamism can be > detected. > What's really needed is to detect the most common case, where objects don't > have unexpected dynamism and can be implemented as hard structures. > > John Nagle > Of course, Guido has left the path to declarations open through the use of function argument annotation. If you wanted to write programs that reasoned about Python programs to optimize them, annotations could come in very useful. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/
From: David Cournapeau on 16 Jan 2010 20:19 On Sun, Jan 17, 2010 at 4:17 AM, John Nagle <nagle(a)animats.com> wrote: > Nobody wrote: >> >> On Fri, 15 Jan 2010 12:34:17 -0800, John Nagle wrote: >> >>> Â Â Actually, no. Â It's quite possible to make a Python implementation >>> that >>> runs fast. Â It's just that CPython, a naive interpreter, is too primitive >>> to do it. Â I was really hoping that Google would put somebody good at >>> compilers in charge of Python and bring it up to production speed. >>> >>> Â Â Look at Shed Skin, a hard-code compiler for Python >> >> A hard-code compiler for the subset of Python which can easily be >> compiled. >> >> Shed Skin has so many restrictions that it isn't really accurate to >> describe the language which it supports as "Python". >> >> Hardly any real-world Python code can be compiled with Shed Skin. Some of >> it could be changed without too much effort, although most of that is the >> kind of code which wouldn't look any different if it was implemented in >> C++ or Java. >> >> The monomorphism restriction is likely to be particularly onerous: the >> type of a variable must be known at compile time; instances of subclasses >> are allowed, but you can only call methods which are defined in the >> compile-time class. >> >> If you're writing code which makes extensive use of Python's dynamicity, >> making it work with Shed Skin would require as much effort as re-writing >> it in e.g. Java, and would largely defeat the point of using Python in the >> first place. >> >> http://shedskin.googlecode.com/files/shedskin-tutorial-0.3.html >> >> If you want a language to have comparable performance to C++ or Java, you >> have to allow some things to be fixed at compile-time. There's a reason >> why C++ and Java support both virtual and non-virtual ("final") methods. > > Â Â My point is that Python is a good language held back by a bad > implementation. Â Python has gotten further with a declaration-free syntax > than any other language. Â BASIC and JavaScript started out declaration-free, > and declarations had to be retrofitted. Â Python has survived without them. > (Yes, there are hokey extensions like Psyco declarations and "decorators", > but both are marginal concepts.) There are efficient implementations of dynamic programming languages which do not rely on declaration (if by declaration you mean typing declaration), even when available: http://strongtalk.googlecode.com/svn/web%20site/history.html See also: http://www.avibryant.com/2008/05/those-who-misre.html David
From: John Nagle on 16 Jan 2010 21:43
David Cournapeau wrote: > On Sun, Jan 17, 2010 at 4:17 AM, John Nagle <nagle(a)animats.com> wrote: >> Nobody wrote: >>> On Fri, 15 Jan 2010 12:34:17 -0800, John Nagle wrote: >>> >>>> Actually, no. It's quite possible to make a Python implementation >>>> that >>>> runs fast. It's just that CPython, a naive interpreter, is too primitive >>>> to do it. I was really hoping that Google would put somebody good at >>>> compilers in charge of Python and bring it up to production speed. >>>> >>>> Look at Shed Skin, a hard-code compiler for Python >>> A hard-code compiler for the subset of Python which can easily be >>> compiled. >>> >>> Shed Skin has so many restrictions that it isn't really accurate to >>> describe the language which it supports as "Python". >>> >>> Hardly any real-world Python code can be compiled with Shed Skin. Some of >>> it could be changed without too much effort, although most of that is the >>> kind of code which wouldn't look any different if it was implemented in >>> C++ or Java. >>> >>> The monomorphism restriction is likely to be particularly onerous: the >>> type of a variable must be known at compile time; instances of subclasses >>> are allowed, but you can only call methods which are defined in the >>> compile-time class. >>> >>> If you're writing code which makes extensive use of Python's dynamicity, >>> making it work with Shed Skin would require as much effort as re-writing >>> it in e.g. Java, and would largely defeat the point of using Python in the >>> first place. >>> >>> http://shedskin.googlecode.com/files/shedskin-tutorial-0.3.html >>> >>> If you want a language to have comparable performance to C++ or Java, you >>> have to allow some things to be fixed at compile-time. There's a reason >>> why C++ and Java support both virtual and non-virtual ("final") methods. >> My point is that Python is a good language held back by a bad >> implementation. Python has gotten further with a declaration-free syntax >> than any other language. BASIC and JavaScript started out declaration-free, >> and declarations had to be retrofitted. Python has survived without them. >> (Yes, there are hokey extensions like Psyco declarations and "decorators", >> but both are marginal concepts.) > > There are efficient implementations of dynamic programming languages > which do not rely on declaration (if by declaration you mean typing > declaration), even when available: > > http://strongtalk.googlecode.com/svn/web%20site/history.html > > See also: > > http://www.avibryant.com/2008/05/those-who-misre.html Yes, that's my point. Psyco was a good first step. The big win with Psyco is that it generally can recognize when a variable is an integer or floating point number, and generate hard code for that. It doesn't do much for the rest of the language. Psyco is really a kind of JIT compiler. Those are useful, but in some ways limited. To go beyond that, global analysis is needed. A big bottleneck in Python is that too much time is spent doing dictionary lookups for things that could be bound at compile time. So the next big win is figuring out which classes definitely don't have any hidden dynamism. A global check is needed to see if any external code messes with the attributes of a class or its functions from outside the function. Most of the time, this is the case. Once that's been done, the class's module can be analyzed for optimization. If the class doesn't use "setattr", etc. to add attributes to itself, then the class can be "slotted", with a C++ like structure for the class members and functions. Global analysis also has to determine the class hierarchy; what inherits from what. It may be necessary to implement "object" as an abstract class with a huge number of virtual functions, so that "duck typing" will work. That's a space cost, but not a time cost. Caller/callee type inference is useful to determine the potential types of parameters. Often, analysis of all the calls to a function will determine the types of many of the paraeters. Then, those parameters can be hard-typed at compile time. You can go this far without the restrictions Shed Skin imposes, such as the restriction that lists must be homogeneous. If you do impose that restriction, array processing becomes much faster. Type inference for array elements is hard when arrays are computed from other arrays, so that's a huge simplification. Yes, you can't use "eval" to get at existing variables. But in Python, you don't really need to. John Nagle |