Recent thread: https://news.ycombinator.com/item?id=33711583

I would recommend calling your project something else. ie, not LuaJIT.

I'd call it Jaunatis (new moon in Lithuanian). A word play on Lua (moon in Portuguese).

Agreed, this has the same "neovim: literally the future of vim" smell to it

Neovim was a fork of Vim so it makes sense to keep the name similar, this project seems to be a completely new codebase.

NeoVim devs earned that 'Neo' by rapidly adding features that were in demand for years (and were subsequently implemented in Vim, in response to NeoVim's new pace of innovation.

Anyway, adding Neo was enough to differentiate the two names, so what's the problem? Let's not forget that Vim did the same thing to Vi when it was named "Vi IMproved."

It's not the name in the neovim case, it's the tagline "literally the future of vim" when vim was still being actively developed by a different group. It is IMO immoral for me to start a site called "Hacker News 2.0: The successor to Hacker News" and start telling people that I'm the next version of it and that's how that comes off to me.

I understand your point, it does seem quite... rude? It reminds me of google calling their programming language "Go" while there was an existing language with that name already.

They didn't call it Vim 9, though. It would be more like if you had a forum called NeoHacker News. Similar to "Hacker News"? Yes. More alliterative? Also yes. Misleading? Potentially not, with the right communication.

There was a similar comment in the other thread (https://news.ycombinator.com/item?id=33712759) with the outstanding (IMHO) suggestion of "Lua DeeJIT" as a nod to the Deegen framework it uses

I’m a bit puzzled by the author’s statement that they’re building a method-at-a-time JIT here.

Sure, that’s an interesting thing you can do and get a Lua JIT, but I don’t see how that is going to be close to the original, tracing LuaJIT, where much of the simplicity (AFAICT) comes from how it creatively punts on several hard backend problems: optimizing in the presence of complex control flow (an linear sequence possibly followed by a loop, that’s all you get on the lowest level), tricky monomorphization decisions (the types you traced are the types you codegen for), eliminating allocations (exiled to trace exits as much as possible), etc.

Where did you learn that? I’ve been wanting to understand LuaJIT for years, but never got around to it.

Is there a “Heart of LuaJIT” style implementation somewhere? Something that’s like a thousand lines but conveys the core ideas.

I think Mike Pall wrote up a whitepaper with the gory details years ago, but I can’t find it atm.

Damn. If you find anything related to it, I’ll happily Venmo you $20. (High roller, I know.)

The magic of making Lua so fast is seriously underrated. I’m fascinated why Lua in particular seems to be so amenable to speedups vs, say, Python bytecode or even Elisp bytecode.

Here are some places to start with, some of his "famous" mailing list posts: https://news.ycombinator.com/item?id=17329902 and 12 year old reddit bullet list of his too: https://old.reddit.com/r/programming/comments/badl2/luajit_2...

Baseless speculation - was it because the VM was so small and well documented?

Never did program much in Lua but really enjoyed the parts where they describe the virtual machine and instruction set.

EDIT:

Link to the paper I mean https://www.lua.org/doc/jucs05.pdf

For a layman like myself this is very approachable. Again I cannot think of another VM language that has anything this small and well documented.

There was a great thread on lambda-the-ultimate where Mike and a some other famous JIT people talk about the benefits of different approaches. I highly recommend searching for those threads.

Can you help me learn how to search for threads on LtU? I tried using the search box for lua but found nothing. Also failed to find anything in the archives.

I use https://hn.algolia.com/ - so let me try - https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

Sorry I read this like 10 years ago.

The other person he talks with is Andreas Gal and they talk about trace trees and method vs tracing jits.

Isn't it the thread that's linked in previous messages in this comment section. Maybe the comment that I linked to in a different message.

I’ll trawl my archives and maybe have a Christmas present for you.

You’re sweet. Thanks for the sentiment, whether or not you find anything. I hope you had a happy Thanksgiving, and that you have a fun and relaxing Christmas break. Cheers.

Lua naturally avoids hashing strings as much as possible. Short strings always have their hash precomputed and local variables are internalized. The default method for implementing variable scopes in dynamic languages just uses a lot of hash tables, but Lua cleverly avoids this:

https://github.com/lua/lua/blob/master/lobject.h#L549

https://github.com/lua/lua/blob/master/lstring.c#L185

Not a language designer, but I have been told the big thing holding back Python is that it exposed too much of its internal guts to external APIs. With so much of the internals exposed, it becomes difficult to make some required backend changes.

Definitely that is true, I wrote a comment about that here: https://news.ycombinator.com/item?id=18041047

Another issue with Python is that the language semantics are much more subtle. This is an excellent talk that shows how even operations that appear simple in Python can be much more complex than they appear: https://youtu.be/qCGofLIzX6g

Not the whitepaper, but here's some resources that I saved for myself to read that might be helpful.

https://www.freelists.org/post/luajit/How-does-LuaJITs-trace...

http://lua-users.org/lists/lua-l/2011-02/msg00742.html

I think I remember the whitepaper, but don't know where to find it these days.

Elisp bytecode now has a compiler using libgccjit, in Emacs 28.

LuaJIT simply did a lot of heavy lifting, implementing a full blown compiler with register allocation and all of that. Lua wasn't special that way. Rather, the Python devs didn't care that much about speed, and Python has the baked in notion that Python programs are a tree of nested dictionaries. Thus, there is only so much performance that could be wrung out of PyPy. But it still helps a lot.

Python 3 was of course a tragedy. If they were going to break stuff at all, they could have made some of that dynamicity optional (few programs use it). Then PyPy could have been the reference implementation and it would be crazy fast, eliminate the GIL, and so on. Just like Lisp implementations have done for 50+ years.

Very interesting to see how it performs against LuaJIT and standard Lua. Would love to see some metrics of it against Luau too.

It’s important to note too that the performance comparison is for the interpreters (not using JIT).

The real tragedy of JIT for games is you can't use it on consoles (AFAIK it's always been forbidden as a threat to security). I'm not sure what the rules are for mobile but I'd be surprised if Apple allowed it either.

Yeah, iOS doesn't allow JITs except for javascriptcore.

And it’s speculated for the same reason.

They’ve loosened up on interpreters, which is nice, but there is no way to make a JIT.

I believe you're permitted to write a JIT that uses JavaScriptCore as its backend. This presumably includes WebAssembly.

...add that to code generation part which means you will have to run a mini compiler + mini linker for each JIT program you run...how wasteful the CPU cycles and how power inefficient that would be

Unless it is an OS service like on Android, Inferno, Oberon, or the big iron computers from IBM and Unisys (where I agree power consumption isn't that relevant).

Which cache the generated code between runs, unless the original code changes, avoid each application shipping their own mini-JIT toolchain, and on Android's case AOT compile the application using the PGO data gathered by the JIT (also shared via PlayStore to other devices), when the device is idle and charging.

> our implementation of the Lua table uses hidden class, instead of a naive hash table as in Lua/LuaJIT.

What is "hidden class"? Is this a data structure?

In languages with dynamic properties on objects, it’s a data structure that keeps track of which properties are present, and how to access them.

Short example In JavaScript: Var a = {}; a.b = 15;

The hidden class object a maps the property b to where its stored, may include type information, etc.

The optimization is usually called shapes outside of JS.

The idea is that shapes (we represent objects like C like structs and not hash maps) are used for relatively non-dynamic objects and then we keep the shape metadata on functions themselves (inline caches) which means property access becomes just a byte offset.

See https://v8.dev/blog/fast-properties