News for hackers. Stuff that matters.
Re:Why Python is Slow: Looking Under the Hood by pnathan on Friday May 09, 2014 @01:06PM (#7721944)
One thing that bothers me - and has for a long time - is why Python (Perl, Ruby, etc), never have really leveraged the work Common Lisp systems have done (CMUCL, SBCL, etc), which provide very good performance without sacrificing dynamic typing or the REPL.
Re:Why Python is Slow: Looking Under the Hood by agentultra on Friday May 09, 2014 @01:40PM (#7722176)
It bothers me too.

Dynamically typed does not imply no types. A good CL implementation takes that to heart and produces really nice code out of the box. And the language spec provides for an API to allow program developers to specify types and hints to the compiler to optimize certain functions and code paths where it's needed.

The benefit from this is that you can rapidly prototype your application and harden it to the platform you deploy on when performance becomes an issue... and with built-in conditional compilation you can do that in a cross-platform way.

Re:Why Python is Slow: Looking Under the Hood by mikeklaas on Friday May 09, 2014 @01:49PM (#7722242)
Python has types; I would even describe it as strongly-typed. It is thus relatively JIT-able (as pypy and psyco have shown).

The chief problem is the high level of dynamic typing going, plus the huge amount of globally-changable assumptions (you can add methods to any instance of a class, you can globally change the definition of language built-ins, etc.)

Re:Why Python is Slow: Looking Under the Hood by agentultra on Friday May 09, 2014 @02:07PM (#7722371)
Some charity, please; I know Python is strongly-typed. It just doesn't take advantage of it and doesn't have facilities for producing fast code.

Common Lisp not only provides type hinting facilities in the language it also provides hinting to the compiler to turn off all run-time type checks in critical code sections.

Re:Why Python is Slow: Looking Under the Hood by chrisseaton on Friday May 09, 2014 @01:51PM (#7722254)
Languages like Python and Ruby have generally looked to the SmallTalk and Self literature (inline caches, dynamic deoptimization, hidden classes etc) rather than Common Lisp, so people aren't ignorant of the research, they're just looking in a different area. If you think there's something in Common Lisp we're missing, point it out!
Re:Why Python is Slow: Looking Under the Hood by pnathan on Friday May 09, 2014 @04:27PM (#7723098)
Well, you are missing compilation to native code. Common Lisp has very good performance characteristics in the hands of an expert, whereas $modern_dynland usually winds up requiring a call out to C, fragmenting the codebase and requiring dual-expertise (one in $dyn_lang, one in C) to maintain effectively.

I don't meant to say USE LISP (but you know, it's available and ready to go!), but I do mean to say: the knowledge (algorithms, research papers, implementations) on how to do this relatively efficiently was developed in the late 70s and early 80s.

edit: SBCL is a good example of what can be done with this knowledge. I would be 100% unsurprised if perl/python/ruby could benefit from these algorithms. See phkoung's blog for what he did with SBCL on numerical analysis and high performance Lisp.

Re:Why Python is Slow: Looking Under the Hood by chrisseaton on Friday May 09, 2014 @04:53PM (#7723205)
But all these algorithms (including compilation to native code) are already in use in JRuby, Topaz, Truffle, PyPy, ZipPy, Unladen Swallow, Pyston, etc. In fact many of these projects use research developed only over the last decade or two, which CL does not use, such as partial evaluation, partial escape analysis, tracing JITs, etc.

So we all know the research, and we know how to put it into practice to make both Ruby and Python run very efficiently. There's no question that Ruby and Python can benefit from this work, because it's already been done many times. What does not look tractable is cramming it all into the existing code bases of CPython or MRI.

Re:Why Python is Slow: Looking Under the Hood by pnathan on Friday May 09, 2014 @05:39PM (#7723387)
I know about those projects too; but at the end of the day, Python is basically CPython. Maybe PyPy should replace Python 2 (I'd argue so, loudly), but for now... CPython is the definition of Python. And it's not a easy problem to make CPython (Python) faster (GIL, effective dynamic analysis), like you said.

What's frustrating is that there's no visible effort to make the Standard Python very fast. Personally, I'd rather see the money poured into Python actually poured into SBCL, but that's just me.

Re:Why Python is Slow: Looking Under the Hood by tshepang on Thursday May 15, 2014 @04:39AM (#7748393)
You saying "I'd rather see the money poured into Python actually poured into SBCL" because CL is a better language?
Re:Why Python is Slow: Looking Under the Hood by dragonwriter on Friday May 09, 2014 @05:18PM (#7723298)
> What does not look tractable is cramming it all into the existing code bases of CPython or MRI.

Of course what is now called "MRI" since 1.9 isn't what was "MRI" before; its what was, previously, one of the alternative implementations (YARV), which got adopted as the mainline implementation.

For some time, there was lots of talk about YARV maybe getting replaced with Rubinius as the engine for the main implementation somewhere down the line, though I think if that happens, its still a few years off. (For one thing, Rubinius will need to get to the point where "Support for Microsoft Windows is coming soon" isn't a thing -- and AFAICT, that "soon" may well be only true on a geological timescale.)

Re:Why Python is Slow: Looking Under the Hood by Amadiro on Saturday May 10, 2014 @09:24AM (#7725300)
AFAIK dynamic languages have gone away from compiling to native code because there is almost no benefit to it, if all you're doing is basically unrolling your interpreter into a unnecessarily huge binary. Methods like tracing JITs are much more effective at delivering good performance in the hotspots of your program.

The exception is when you allow yourself to annotate your code with type information, which allows you to eliminate a lot of the code you'd otherwise emit. Python for instance allows you to do this with Cython/snakeskin/etc., so the technique (that I think) you're talking about is applied there, because it is effective.

Re:Why Python is Slow: Looking Under the Hood by ScottBurson on Friday May 09, 2014 @08:00PM (#7723919)
And the easiest way to leverage that effort is to use CL as an implementation platform. Whatever you think of CL as a language to program in directly, it really shines as a language to implement dynamic languages in. You get a lot of useful pieces, including native-code compilation, some very powerful control-flow operators, multithreading (in most implementations), and of course garbage collection, for free.

There's a more subtle benefit as well. Because compilability was a key goal in the design of CL, using it as a translation target while you're designing a new dynamic language will help you make sure your language is also efficiently compilable.

EDITED to add: in CL, the whole tagged-pointer thing has been taken care of for you, so you don't have to think about it (cf. [0]). This means you get fixnums (unboxed immediate integers), with automatic overflow to bignums (arbitrary-precision integers), for free. In fact there's the whole CL numeric tower, including ratios and complex numbers. Other goodies: multiple inheritance. Multimethods (multiple dispatch). Class creation and redefinition at runtime.

It baffles me that anyone would want to reimplement all that stuff, or even part of it.

[0] https://mail.python.org/pipermail/python-dev/2004-July/04614...

Re:Why Python is Slow: Looking Under the Hood by jules on Friday May 09, 2014 @06:58PM (#7723705)
Python is a very different language than Lisp. It's much more dynamic. Every method call conceptually involves a hash table look up. Lisp on the other hand is much closer to the bare metal, and much easier to implement efficiently. For Python the implementation techniques of Self would be more appropriate.
Re:Why Python is Slow: Looking Under the Hood by lispm on Friday May 09, 2014 @07:33PM (#7723843)
> Every method call conceptually involves a hash table look up

Like CLOS.

Re:Why Python is Slow: Looking Under the Hood by WaxProlix on Friday May 09, 2014 @01:46PM (#7722224)
What work specifically are you referencing?
Re:Why Python is Slow: Looking Under the Hood by agentultra on Friday May 09, 2014 @01:56PM (#7722292)
Re:Why Python is Slow: Looking Under the Hood by pnathan on Friday May 09, 2014 @04:24PM (#7723089)
JIT compiling to native code from the REPL is a big one.