News for hackers. Stuff that matters.
Re:SPy: An interpreter and compiler for a fast statically ty... by cardanome on Wednesday November 05, 2025 @06:05AM (#45821601)
Common Lisp also allows you to redefine everything at runtime but doesn't suffer from the same performance issues that Python has, does it?

Doe anyone have insight into this?

Re:SPy: An interpreter and compiler for a fast statically ty... by brabel on Wednesday November 05, 2025 @07:50AM (#45822291)
Common Lisp is not a runtime, it’s a specification. Implementations are free to compile everything to fast native code, or to interpret everything. Various available implementations do that and everything in between. That said , SBCL and the commercial implementations can be extremely fast, especially if you specify types on tight loops. SBCL comes with a disassembler that shows you right in the REPL the Assembly a function compiles to so you can even get close to C performance.
Re:SPy: An interpreter and compiler for a fast statically ty... by pjmlp on Wednesday November 05, 2025 @09:57AM (#45823531)
Addedum that having a disassembler is quite common language primitive in most compiled Lisps, since early days.
Re:SPy: An interpreter and compiler for a fast statically ty... by a-french-anon on Wednesday November 05, 2025 @07:01AM (#45821929)
Common Lisp doesn't use (expensive) CLOS dispatch in the core language, e.g. to add two numbers or find the right equality operator. That's one known pain point due to CLOS having been "bolted-on" rather than part of the language which makes the divide between internal (using typecase and similar) and external (generic functions) dispatch pretty ugly; and gave use the eql/equal/equalp/etc... hell.

Thing is that you need a complex JIT like Julia's or stuff like https://github.com/marcoheisig/fast-generic-functions to offset the cost of constant dynamic dispatch.

I actually had such a conversation on that comparison earlier this year: https://lwn.net/Articles/1032617/

Re:SPy: An interpreter and compiler for a fast statically ty... by psychoslave on Wednesday November 05, 2025 @07:38AM (#45822204)
What is CLOS in this context?
Re:SPy: An interpreter and compiler for a fast statically ty... by bradrn on Wednesday November 05, 2025 @07:42AM (#45822237)
Re:SPy: An interpreter and compiler for a fast statically ty... by brabel on Wednesday November 05, 2025 @07:43AM (#45822245)
Common Lisp Object System. The language amazing version of what OOP can be.
Re:SPy: An interpreter and compiler for a fast statically ty... by martinflack on Wednesday November 05, 2025 @09:26AM (#45823194)
> and gave use the eql/equal/equalp/etc... hell.

You don't like those? I've always considered them a fairly elegant deconstruction of the problem domain of equality checking. DWIM languages can get very confusing when they DWIM or don't DWIM.

Re:SPy: An interpreter and compiler for a fast statically ty... by a-french-anon on Wednesday November 05, 2025 @10:20AM (#45823774)
Where's the elegance? Equality is well defined on everything handled by those three, since they only compare the same types without doing coercion. Plus, you can't extend these to handle user types.

Which is why https://cdr.common-lisp.dev/document/8/cleqcmp.html exists, really; all the "copy-x" would benefit from the same fix, in my opinion.

Re:SPy: An interpreter and compiler for a fast statically ty... by majoe on Wednesday November 05, 2025 @12:54PM (#45825785)
> Thing is that you need a complex JIT like Julia's or stuff like https://github.com/marcoheisig/fast-generic-functions to offset the cost of constant dynamic dispatch.

Julia is always the odd one out, when talking about dynamic vs. static dispatch, because its JIT compiler acts more like an Ahead-of-Time compiler in many regards.

In the best case, types are statically decidable and Julia's compiler just produces a static dispatch and native code like e.g. a C compiler would.

In the worst case, there are a big (or unlimited) number of type candidates.

The grey area in between, where there are a limited number of type candidates, is interesting. As far as I understand, Julia does something similar to the link you provided. Based on some heuristics it will compile instances for a "sealed" number of candidates and fallback to a fully dynamic dispatch, if there are two many type candidates.

At JuliaCon 2025 there was an interesting talk about this topic: https://m.youtube.com/watch?v=iuq534UDvR4&list=PLP8iPy9hna6S...

For the worst case scenario, Julia chooses what's in my regard the nuclear option: If the types are not decidable, it just ships the whole compiler with your code and tries again at runtime. But I guess, that's not the only possible solution. Presumably, it would also be possible to fallback to a Julia interpreter for dynamic code. That would be more similar to what JavaScript is doing, just the other way around. Instead of interpreting the majority if the code and optimising hot paths with a JIT, our alternative Julia would compile most code statically and use the interpreter for the dynamic parts.

Re:SPy: An interpreter and compiler for a fast statically ty... by pjmlp on Wednesday November 05, 2025 @06:45AM (#45821825)
Just like Smalltalk and SELF, also Lisp Machines and Interlisp-D.

Usually comes down from a urban myth that Python is special and there was no other dynamic language before it came to be.

The JIT research on those platforms is what gave us leading JIT capabilities on modern runtimes, OpenJDK HotSpot traces back to Smalltalk and StrongTalk, while V8 traces back to SELF.

Especially in Smalltalk and SELF, you can change anything at any time across the whole image, and have the JIT pick up on that and re-optimize.

Granted what messes up Python, or better said CPython implemenation, is that C extensions are allowed to mess up with its internals thus making void many possible optimizations that would be otherwise available.

A reason why JVM, CLR, V8, ART make use of handles and have marshaling layers not allowing such kind of liberties with native extensions.

Re:SPy: An interpreter and compiler for a fast statically ty... by gjvc on Wednesday November 05, 2025 @07:47AM (#45822269)
Great explanation. Five years ago I did the genealogical work to discover that StrongTalk begat HotSpot (by virtue of having some of the same authors) It was quite a joy to discover!
Re:SPy: An interpreter and compiler for a fast statically ty... by vidarh on Wednesday November 05, 2025 @10:37AM (#45824018)
> Granted what messes up Python, or better said CPython implemenation, is that C extensions are allowed to mess up with its internals thus making void many possible optimizations that would be otherwise available.

I'm doing a Ruby compiler (very, very slowly, though faster again now with Claude Code doing most of the heavy lifting), and the same issue with Ruby has made me seriously toy with the idea of one day embedding a C compiler so that the Ruby compiler can optimise across both (it'd still have to deal with linking to third party C code, of course, which is one reason I'm hesitating) as a simple not-so-optimizing C compiler is like a trivial toy compared to compiling Ruby where just the parser makes you want to claw your eyes out, but it'd at least widen the surface a bit.

Re:SPy: An interpreter and compiler for a fast statically ty... by pjmlp on Wednesday November 05, 2025 @02:01PM (#45826517)
Have you seen how TruffleRuby makes use of LLVM bitcode, to ship C compiler as well?
Re:SPy: An interpreter and compiler for a fast statically ty... by vidarh on Wednesday November 05, 2025 @06:06PM (#45829285)
I haven't, but not surprised given some of the very aggressive (in a good way) things they've done to address performance. It's impressive. I spoke to Chris Seaton back when it was getting started, and loved many of the things they were doing. Personally I want something leaner, and self-hosting, but that's also why mine is still experimental and wildly incomplete and TruffleRuby works for a lot of things.
Re:SPy: An interpreter and compiler for a fast statically ty... by pjmlp on Thursday November 06, 2025 @03:09AM (#45832719)
Here, I don't know all the details, just remember having seen some talks about this in the past.

It builds on top of this effort, https://medium.com/graalvm/graalvm-llvm-toolchain-f606f995bf

"From Interpreting C Extensions to Compiling Them"

https://eregon.me/blog/assets/research/from-interpreting-c-e...

Re:SPy: An interpreter and compiler for a fast statically ty... by vidarh on Thursday November 06, 2025 @04:53PM (#45840926)
Thanks, will definitely give it a read.