One comment on Julia though. It isn't like other JIT languages. It's a lazy statically compiled language with a REPL. It uses LLVM (like Clang) to compile functions the first time they're called for a given set of input types (every function is like a C++ template by default). This makes it easy to get C-level speed, because all it is is a different front end (with extensive type inference infrastructure/default "auto" for all types) to the same back end.
This is in constrast to an interpreted language, where code only gets compiled when hot (but then with the potential of profile-guided compilation, which could sometimes let them run even faster).
This sounds a lot like Object.freeze[0] in JS, at least for prototypes. I can't remember any of the major engines doing this, but it could be an interesting direction to explore.
[0]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
Common Lisp, well, I suppose I'll leave that up to the reader. Personally, I observe that in a context where dynamic languages are being discussed in a positive manner, Common Lisp is a dynamic language, but when static languages are being discussed in a positive manner, suddenly Common Lisp is a static language. (In fact, that pattern may hold in general; for any given programming individual language characteristic being positively discussed, Common Lisp will have it, no matter how contradictory the whole set may be.) Under the hood, it must resemble one or the other more strongly, but after years of consuming advocacy, heck if I know which.
Can't speak to the other two.
a) fast incremental development of Lisp code. For that one got Lisp interpreters (executing the code on the s-expression level) and quick incremental compilers. Code then tends to be more on the dynamic side with full debug information.
b) delivery of optimized applications for deployment. We got optimizing native-code AOT compilers (using type hints and type inference), block compilers (optimizing larger batches of code), whole-program Lisp-to-C compilers and application delivery tools with tree shakers. Code then tends to be more on the static side with little or not debug/development information.
Between a) and b) there is a continuum of different approaches - often in the same program in different code sections - where compilers may support different code generation strategies.
Thus one can develop using an interpreter or fast incremental compiler, and then deliver with an optimizing compiler.
When old timers discuss Common Lisp, usually it goes around the development experience of yore that Allegro and LispWorks still offer, but most FOSS variants fail short of.
As for Julia, yes their main niche is scientific computing. Being able to write everything 100% in Julia, instead of switching to C, C++ or Fortran.
The adoption rate seems to be "slow and steady".
CL as a language is essentially untyped[1], but does provide some provision for type hints in the standard. Some implementations extend this to the point where they can generate code as efficient as a typed language. Most implementations can also use these to detect some fraction of type errors when compiling.
Some features in lisp were designed with less flexibility to allow for improved performance while others with flexibility at the expense of performance.
I can't imagine how anyone could ever consider CL to be a static language without horribly misunderstanding the term, the language, or both.
1: You can declare types of variables, but the specification clearly intends this to be for optimization purposes: " Type declarations present in the compilation environment must accurately describe the corresponding values at run time;..." the fact that some implementations use these type hints for more does not IMO change the nature of the language.
If you look at the file compilation spec in the standard, the compiler is allowed to inline functions and it can assume that a function in the same file can't be redefined later. -> no late binding for the calls in the file. A function declared by the developers to be inlined, makes them also no longer use late-binding for calls.
Specific implementations may also remove the compiler (see Allegro CL, LispWorks and some others), parts of an interpreter and other stuff of a typical Lisp runtime from a delivered applications. Generally function calls then might no longer use late-binding.
The CL standard might mention some dynamic facilities, but actual implementations might remove parts of that from a delivered application, too.
Example for more static facilities were the block compilation mode of Lucid CL, IIRC CMU CL also had a block compilation mode, some compilers do their own inlining (Allegro CL), mocl/clicc (and some others) do 'static' compilation of whole-programs for a large CL subset to C applications, LispWorks has an extensive delivery system which removes various unused Lisp facilities from an application ( http://www.lispworks.com/documentation/lw71/DV/html/delivery... ) and which can drastically reduce dynamic features.
Common Lisp as a standard is highly dynamic. Common Lisp as actually implemented and used is sometimes much less dynamic.
Does the existence of Cython mean that Python is a static language? Most python programmers will never use it so I would argue not.
The CL standard is describing more 'static' options than usual for Lisp. Lexical binding is more static than Dynamic Binding. Macros are more static than Fexprs. A file compiler is more static than an interpreter.
Common Lisp was already designed with support for higher-performance implementations on stock hardware.
Python is not a 'language' - but for most people it is the combination of a language and its implementation. In this Python one of the main (THE main?) ways to get better performance is to write the code in C.
Not so Common Lisp - it is a language and a bunch of more or less conforming implementations. There the first and most important way to get better performance always has been to use an optimizing to-C or a to-Machine-Code compiler, code hints and compiler switches (SPEED, SAFETY, DEBUG, ...).
For example the developers of a large CAD system written mostly in Lisp (iCAD, PTC, ...) did not compete with Python at that time, but with C++ based CAD systems. Having a dynamic CLOS implementation is then often a burden: how many objects can be created per second? How many generic function calls are possible per second?
For example LispWorks allows the slot-value function to call a generic function slot-value-using-class. This is slow. Thus by default it is shut off.
Clozure CL has I/O functions over streams. By default streams are not CLOS objects / functions. One can get them as CLOS objects, but one has to do that.
There are many other examples where the implementation provides a dynamic version and a more static version.
The typical example in Common Lisp, the language itself: structures and CLOS classes.
Structures are efficiently compiled with inline access, no slot information, no updates on structure changes, efficient access into the slot (-> no multiple inheritance with changing slot otder).
CLOS classes OTOH are the opposite. More dyanamic, especially in an implementation with the Meta-object Protocol supported.
Whenever you decide to use structures in your program, then you choose 'static & fast'. When you choose CLOS classes then you choose: runtime dynamism and generally slower.
The standard itself does not talk about a specific implementation - the implementations in use and the needs of the developer for their application delivery determines then how 'static' the implementation can be or needs to be.
I think "unknown type at compile time" and "JIT-able constructs" are opposites. You'd end up with a tracing JIT that compiles the same set of code in different, specialized ways based on runtime determination of the the input types as though the lang was static. Which is basically what we have today. You can have optionally typed languages, but you still have to write your runtime and compiler to the lowest common denominator, so you probably want to go more strict if you care about your (runtime or compile-time) compiler and runtime performance (as the Dart devs learned IIRC).
I would assume "JIT-ability" is proportional to the immutable input and output knowledge of the code to be compiled.
Not opposites exactly. Types enforce a phase separation between compile time and runtime, but there's no reason you can't have a compilation phase at runtime too. This leads to staged programming, like MetaOCaml.
It's kind of like how some static languages use code generation, say, based on a database schema, which is then compiled, except this would happen at runtime. It would have to let you programmatically guide this process dynamically, but the result is very close in performance to a statically compiled language.
Someone did a study of their Python servers, and found that after an initial phase, almost no dynamic facilities were used at all!
it seems to me that it would also be feasible to go the other way, and allow users to define things as structs, and if you want to offer hash-like access it would be easy to lay down hash-like access to the struct members
I like that. Smalltalk kind of did this, but much less. Objects were just arrays of object references, but certain object references were adorned smallint values. People also wrote hash-like accessor implementations for objects. "Shape changes" to objects of a certain class would require a stop the world re-allocation of all such objects, however.
The Ruby community went in different direction here [2] with RTL MJIT.
[1] https://github.com/eclipse/omr [2] https://medium.com/square-corner-blog/rubys-new-jit-91a5c864...
What it just got recently is a pluggable specializer that allows more interesting optimizations to be written in a higher level language (NQP). Since this is closer to Perl 6 than the VM it has more information with which to decide how to optimize code. (Note that it also means you can do the same with any other language that runs on MoarVM, even if it is very different from Perl 6)
Even LuaJIT, which is to my understand the closest anyone has come to this, still was retrofitting JIT onto an existing language.
I'm not convinced we're going to see much more performance out of JS, for instance, which is about as fast as a dynamic language can go nowadays. But I wonder what the real limits of a "dynamic scripting" language would be from this perspective.
Edit: Per my other comment about indirection being a performance poison pill, here's an example of an idea where a new scripting language might be able to get a lot of performance. Suppose you keep the ability to dynamically create classes and load code and so forth, so that (just as an example, not necessarily a good idea) you can write code that dynamically connects to a database and loads in tables as classes with automatically defined properties, etc. But instead of working the way the languages do now, instead of constantly walking through all the layers of indirection that can be used to implement all this at every call site and for every call, what if you could do something like the "pledge()" call that says "OK, this is it, I'm all set up, the dynamism is all done, you may now assume that all the type analysis you've done is now complete". Now the JIT can drop all of its paranoia code. It isn't completely obvious to me how to do this correctly (by implication, if you can "connect to a database" before this pledge()-like call is done, you have to have a pretty complete runtime available just for that), nor is it obvious to me how this would affect the type system, etc. Maybe it's not possible. But it's the type of thing I mean that would be interesting to have examined by someone, who might be able to concretely show why it's not possible, or, whoknows, make it work. And that's just one idea of how to design a dynamic language from the top for performance, basically off the cuff; who knows what a smart person who sat down and thought about this for a couple of weeks before even beginning to code could come up with.
Another one is that it isn't obvious to me that scripting languages must be all based on hash tables that laboriously can be cast back to structs by the JIT; it seems to me that it would also be feasible to go the other way, and allow users to define things as structs, and if you want to offer hash-like access it would be easy to lay down hash-like access to the struct members (iteration, etc.), and either implicitly or explicitly also offer an "overflow" hash table if desired. This would give at least a bit of locality control. Something something arrays too for array control, and suddenly you're starting cook with gas.