News for hackers. Stuff that matters.
Re:Why Python Is Slow: Looking Under the Hood by ggchappell on Tuesday November 18, 2014 @02:51PM (#8625915)
> 2. Python is interpreted rather than compiled.

Can we stop saying things like this? Virtually all Python is compiled, as part of the interpretation process.

There is a valid point here, of course. The point is that the top Python compilers do not compile to native code. So say that.

Words have meanings. Use them correctly. <grumble, grumble>

Re:Why Python Is Slow: Looking Under the Hood by jeffreyrogers on Tuesday November 18, 2014 @03:09PM (#8626019)
There are three primary methods of running code: interpretation, compilation to object code, and running on a VM/JIT. Obviously python is interpreted by any meaningful definition of interpretation and thus it is not compiled in the typically sense of the word (at least not in the implementation everyone uses).

The reason human language is so expressive is because we can leave out a lot of context and formalism that is required in mathematics and programming languages because the listener/reader will be able to infer it. It's pedantic to expect a writer to hedge against every possible interpretation, when their focus should be on communicating clearly in the first place.

Re:Why Python Is Slow: Looking Under the Hood by ggchappell on Tuesday November 18, 2014 @09:02PM (#8627788)
> It's pedantic to expect a writer to hedge against every possible interpretation, when their focus should be on communicating clearly in the first place.

Our discipline (computer programming) is, perhaps more than any other, one in which precision of meaning is of paramount importance. I expect people to hedge. Especially when it's easy: "Python is not usually compiled to native code."

In any case, since Python is virtually always compiled, saying it's not is simply false. That's hardly clear communication, I think.

> Obviously python is interpreted by any meaningful definition of interpretation ....

Now, here I'm not sure what you mean. You've already distinguished interpretation from Just-In-Time compilation to Virtual Machine code. Since the latter is pretty much always what happens with Python, would you then say that Python is not interepreted? (Serious question!)

Me, I don't have a problem with these concepts overlapping (and I don't see why many people do). Python is interpreted. The usual mechanism for this is JIT compilation to virtual-machine code, which is then -- in the case of CPython -- interpreted as-is. In contrast, something like "C" is traditionally Ahead-Of-Time (AOT) compiled to native code. Interpreted: no. Compiled: yes. And then there are languages whose interpreters do not involve a compilation step -- although they are becoming rare.

Re:Why Python Is Slow: Looking Under the Hood by andreyf on Tuesday November 18, 2014 @03:28PM (#8626123)
FWIW, wikipedia says "The main Python implementation, named CPython, [...] compiles Python programs into intermediate bytecode, which is executed by the virtual machine." I'm not sure why people call Python's VM an interpreter, but it's definitely interpreting byte code, not the source directly.

This is very different from Perl, where a line of code isn't parseable until you know the values of the variables, e.g.

    whatever / 25 ; # / ; die "this dies!";
Re:Why Python Is Slow: Looking Under the Hood by haberman on Tuesday November 18, 2014 @03:53PM (#8626276)
> This is very different from Perl, where a line of code isn't parseable until you know the values of the variables

Perl is crazy, but I don't think it's that crazy.

From perlcompile(http://perldoc.perl.org/5.8.9/perlcompile.html)

    Perl has always had a compiler: your source is compiled
    into an internal form (a parse tree) which is then optimized
    before being run.
If what you say is true, it would not be possible to produce a parse tree prior to running the program.

I could not get your code sample to die based on the type/value of "whatever" -- can you?

Re:Why Python Is Slow: Looking Under the Hood by scott_s on Tuesday November 18, 2014 @04:09PM (#8626364)
Apparently, Perl is that crazy. See https://news.ycombinator.com/item?id=5770531

Basically, you can't always parse a Perl program without running it. There is a subset that you can, not not everything.

Re:Why Python Is Slow: Looking Under the Hood by haberman on Tuesday November 18, 2014 @04:26PM (#8626459)
> Basically, you can't always parse a Perl program without running it.

That is true, but what the grandparent said is not: https://news.ycombinator.com/item?id=8626454

To expand on this, I think a succinct way of summarizing the issue is: you might have to run BEGIN blocks in Perl to parse the non-BEGIN-block part of the program. The canonical example of this is:

    BEGIN {
        if(arbitrary_function()) {
            eval "sub whatever() { }; 1" or die $@;
        } else {
            eval "sub whatever { }; 1" or die $@;
        }
    }

    # The parsing of this line depends on the result of arbitrary_function()
    whatever  / 25 ; # / ; die "this dies!";
arbitrary_function() can be anything, so the only way to parse the rest of the program is to actually run arbitrary_function().
[deleted]
Re:Why Python Is Slow: Looking Under the Hood by haberman on Tuesday November 18, 2014 @04:25PM (#8626454)
I am aware of this result. However, that differs from what GP said in one important way.

Grandparent said "This is very different from Perl, where a line of code isn't parseable until you know the values of the variables."

If that were true, it would imply that you could have a loop whose body is parsed differently on every iteration. It would imply that the parser runs inline with the interpreter. Neither of those things are true.

What your link says is that the parsing can vary based on whether or not a subroutine prototype with appropriate arity was previously defined. That is a very different thing. A statement in a loop could be parsed differently depending on what was defined before it, but that parse doesn't change just because a subroutine is defined later, I'm pretty sure.

Yes, this means Perl's parsing is undecideable, because you can conditionally define a subroutine (this might be limited to BEGIN blocks; I'm not sure). But that's not the same as saying the parse varies based on the "values of the variables."

Re:Why Python Is Slow: Looking Under the Hood by Moto7451 on Tuesday November 18, 2014 @04:11PM (#8626380)
Did you try running that?

  [13:06:36] ~$perl -e 'whatever / 25 ; die "this dies!";'
  this dies! at -e line 1.
  [13:06:36] ~$

  [13:06:38] ~$perl -e "use strict; use warnings; whatever / 25 ; die "this dies!";"
  Useless use of division (/) in void context at -e line 1.
  Bareword "whatever" not allowed while "strict subs" in use at -e line 1.
  Execution of -e aborted due to compilation errors.
  [13:06:38] ~$
After removing the comment the former parses just fine and died with "this dies!". The latter parsed and found a syntax error.
Re:Why Python Is Slow: Looking Under the Hood by twoodfin on Tuesday November 18, 2014 @04:00PM (#8626322)
FWIW, wikipedia says "The main Python implementation, named CPython, [...] compiles Python programs into intermediate bytecode, which is executed by the virtual machine." I'm not sure why people call Python's VM an interpreter...

Because CPython implements its virtual machine with an interpreter, without the option of, say, a JIT compiler.

Re:Why Python Is Slow: Looking Under the Hood by bunderbunder on Tuesday November 18, 2014 @03:03PM (#8625977)
Might want to start by asking the core Python community. Check out the first sentence on wiki.python.org.

Nowadays it's the norm for interpreted languages to be JIT-compiled. They still universally call themselves interpreted languages because the jitter is an implementation detail, and the use of one does nothing to negate the characteristics they're trying to advertise when describing themselves as interpreted languages.

Think of it this way: Can I take a text file containing source code in the language and execute it anywhere that has a special program for executing files written in that language (commonly known as an interpreter)? Or do I have to manually perform an intermediate step of converting the program to some binary representation using a special program for doing that (commonly known as a compiler), and then execute the output that program produces?

Re:Why Python Is Slow: Looking Under the Hood by lispm on Tuesday November 18, 2014 @03:17PM (#8626056)
There are several compiler-only Lisp implementations which can execute source from text. Nobody there would call them interpreter because of that. In Lisp we call them interpreter, when the implementation traverses the source code during execution. If it compiles the source to some byte-code or machine code, we call in Compiler.

The incremental nature of a compiler, then does not make it that we would call it an Interpreter. We would call it an incremental compiler. That's a compiler, which can compile all expression, regardless of the size of the expression, not just 'whole programs'.

Re:Why Python Is Slow: Looking Under the Hood by ggchappell on Tuesday November 18, 2014 @09:00PM (#8627783)
> There are several compiler-only Lisp implementations which can execute source from text.

Why not? I would.

Re:Why Python Is Slow: Looking Under the Hood by ggchappell on Wednesday November 19, 2014 @03:45PM (#8632506)
Oops. My "Why not?" above is in reference to this:

> Nobody there would call them interpreter because of that.

Re:Why Python Is Slow: Looking Under the Hood by lispm on Thursday November 20, 2014 @05:16PM (#8638514)
Why would you call an 'compiler' an 'interpreter'?

The following is Clozure Common Lisp. Every single input is compiled directly to machine code. Feed it source text and everything is compiled. Each and every expression.

Why should I call a direct native code compiler an 'Interpreter'? Makes no sense.

    ? (let ((f (lambda (a)
                 (+ 1 a))))
        (disassemble f)
        (funcall f 2))


    ;;; (lambda (a) (+ 1 a))
    L0
        (leaq (@ (:^ L0) (% rip)) (% fn))       ;     [0]
        (cmpl ($ 8) (% nargs))                  ;     [7]
        (jne L57)                               ;    [10]
        (pushq (% rbp))                         ;    [12]
        (movq (% rsp) (% rbp))                  ;    [13]
        (pushq (% arg_z))                       ;    [16]

    ;;; (+ 1 a)
        (leaveq)                                ;    [17]
        (testb ($ 7) (% arg_z.b))               ;    [18]
        (jne L39)                               ;    [22]
        (addq ($ 8) (% arg_z))                  ;    [24]
        (jo L32)                                ;    [28]
        (repz)
        (retq)                                  ;    [30]
    L32
        (jmpq (@ .SPFIX-OVERFLOW))              ;    [32]
    L39
        (movl ($ 8) (% arg_y.l))                ;    [39]
        (jmpq (@ .SPBUILTIN-PLUS))              ;    [44]

    ;;; #<no source text>
    L57
        (uuo-error-wrong-number-of-args)        ;    [57]

    3 ; result
Re:Why Python Is Slow: Looking Under the Hood by ggchappell on Thursday December 04, 2014 @03:51PM (#8701548)
An interpreter is a box that takes code in some programming language and executes it. A compiler is a box that takes code is some programming language and translates it into another programming language.

Traditionally, the twain ne'er did meet. However, many (most?) modern interpreters include a compiler as part of their implementation. Using a compiler in this manner is "Just in Time" (JIT) compilation.

So a box that takes Clozure Common Lisp and converts it to machine code is a compiler.

A box that takes Clozure Common Lisp, feeds it into this compiler to obtain machine code, and executes this machine code immediately, is an interpreter, using JIT compilation to do its interpretation.

Re:Why Python Is Slow: Looking Under the Hood by lispm on Thursday December 04, 2014 @04:13PM (#8701668)
> An interpreter is a box that takes code in some programming language and executes it.

The interpreter executes that language, not another.

> A box that takes Clozure Common Lisp, feeds it into this compiler to obtain machine code, and executes this machine code immediately, is an interpreter, using JIT compilation to do its interpretation.

It's neither an interpreter not using JIT compilation. It's an AOT compiler. It's just an incremental compiler.