> I said a static compiler doesn't make sense for a dynamic language (saying you can't do it is tautological, it is like trying to get dry water).

Not at all, dynamically typed languages have varying amounts of effective dynamicity (and staticity), some should be static enough to infer most types statically. Erlang for instance is not overly dynamic.

> I am talking about dynamic compilation to machine code (Not JIT). From that, you can alter how much code in-lining and optimization happen in nested calls. It is a much used technique and I do not need to prove its validity.

You're describing JITs here, why are you saying "not JIT"?

> Everyone in here seems blind to the possibility, which puzzles me.

Everyone "seems blind" because you're describing JITs and saying you're not talking about JITs, you're about as clear as tar during a moonless night here.

No, JIT is a specific type of dynamic compilation. It is not every type of dynamic compilation. Maybe I mean 'incremental compilation.'

I am not describing JITs, I am describing VM based languages, which have the ability to incrementally statically compile functional objects. Does that help?

Then the confusion probably comes from the fact that Python's main implementation is VM based. So suggesting what they are already doing as an improvement over what they are already doing is confusing to say the least. Perhaps they need a better VM, but that is the technique they use. To see Python's byte code open up a .pyc file.

> Perhaps they need a better VM, but that is the technique they use. To see Python's byte code open up a .pyc file.

Ohyes is talking about per-function static compilation performed on the fly to machine code. Not bytecode.

It seems about halfway between static compilers and JITs really: functions are compiled to actual machine code statically, but the VM can recompile functions or compile new functions and replace old ones (of the same name) on the fly, e.g. during a REPL session.

That's not what Python does, Python code is compiled to VM bytecode and the VM does not compile it any further.

Under ohyes's scheme, the VM would compile that bytecode further down to machine code (or just skip the bytecode). It's closer to what HiPE does than what Python does.

Yes, this is correct.

Yes.

Do you mean AOT compilation? From an earlier post I gleaned that the Uladen Swallow project used the LLVM backend, which is variously described as a Jit, Ahead of Time compiler, incremental compiler and various other things. It's clear there is some confusion in language, but I got what you are talking about.

At any rate, if I am reading the earlier post, it was tried and not found to be effective. This surprises me greatly. LLVM is of the highest quality and very fast. I'd love to know why people considered it to have gone "wrong" when it came to Unladen Swallow Python.

Good call, I think you are right. My intended point was that the OP was dismissive of the idea of Incremental/AOT compilation as a possibility. I was not terribly clear and may have misread him.

The idea for LLVM is that you can target the LLVM IR or LLVM Byte-code, and LLVM will provide the platform for your regrettable compiler. It has both a JIT and a native compiler component. You can run the AOT compiler either as incremental or sucking in a bunch of source and doing C style static compilation.

I am by no means an expert obviously, but when I evaluated it for a project it seemed to be geared towards generating fast code for C/++ like languages... for which you tend to know the machine types of things, and be operating in terms of machine floats/doubles/integers/etc. Which doesn't seem to be much of a problem for Python, honestly.

The 'virtual machine' is more of a Bytecode model... as the name implies, it is low level). You would have to build your own virtual machine (PythonVM or what have you) on top of it. This would need to be a complete VM with the ability to generate LLVM bytecode. Then you could take advantage of the SSA transforms constant reduction and other nice parts of the LLVM (peephole optimization for example).

But I guess the point is, LLVM takes care of one hard part for you, but there are a bunch of other difficult parts which would still need to be handled. Particularly the garbage collector. I'm sure unladen swallow generated code is bleeding fast because of its use of LLVM.

All of this said, I'm pretty sure that the project died with Python 3. Maybe this whole discussion is missing the point entirely? How do you write a fast compiler for a language which has no standard? It is bound to change unpredictably and be an incredibly frustrating task.

JIT is dynamic compilation to machine code. Feel free to explain why your technique is not JIT, though.

This is not a JIT.

http://paste.lisp.org/+2N29