atgreen

Hey, I'm the author. Yes, I worked in the University of Toronto's AI lab as a student many years ago, back when Lisp in AI was a thing. It definitely would be full circle!

I worked on this years ago. Gcj did support loading and interpreting bytecode alongside the AOT-compiled code. libffi's closures were originally implemented to support calling interpreted methods from AOT-compiled code, because a class's vtable could include a mix of AOT-compiled and interpreted methods. You could also compile jar files directly into .so files, which would get loaded and used at runtime if a classloader ever loaded the corresponding jar.

VISA open sourced an expense-to-run harness they developed under glasswing, and I used that as the basis of a skill that I've found to be pretty effective with Claude Code: https://github.com/atgreen/secscan-skill

TBH, the complexity of this step grew over time, and the overhead snuck up on us. The prep step does useful work (eg. determine stack space requirements). It's just that we don't have to do it again.

Something I should have mentioned is that we could have avoided the new APIs if only there was space in the ffi_cif to stash a plan pointer. And I didn't want to break ABIs for this.

Yes, that's part of what was done here. So, create a plan, and then for some subset of plans, create AOT-compiled templates. The analogies are: a) original implementation is like interpreting via walking a syntax tree b) building/caching an execution plan is like interpreting by executing bytecode generated from the syntax tree c) using an AOT-compiled template is like execution from qemu's old TCG template system But we only do (c) for a popular subset of function signatures. The biggest win was (b), but (c) is still an improvement over (b).

Here's another option.. I created an optimizing eBPF compiler in Common Lisp for a lisp-ish DSL. It's nice because you can compile and load your eBPF code all in-process in lisp (even from your REPL) without any external tooling. https://github.com/atgreen/Whistler

No, they are different. I just bundled them together for convenience in this POC. The only real thing in common is that they both use eBPF.

Have a look at https://github.com/atgreen/rhel-block-copyfail

Don't disagree, but there are eBPF mitigations that work as alternatives to unloading kernel modules.

I created something similar earlier today: https://github.com/atgreen/block-copyfail

This is not my experience. I've been experimenting with something very similar to vera. However my language transpiles into multiple languages (Java, Typescript, Common Lisp, Rust, C++, Python, C# and Swift). The transpiler is written in the language itself (there's a separate bootstrap transpiler written in Common Lisp). But where I'm going is that Claude, at least, is extremely capable at writing decent code in my new language with barely any prompting; just minimal guidance on the language itself and no examples.

I enjoyed reading this. Thank you for sharing.

I learned Common Lisp years ago while working in the AI lab at the University of Toronto, and parts of this article resonated strongly with me.

However, if you abandon the idea of REPL-driven development, then the frontier models from Anthropic and OpenAI are actually very capable of writing Lisp code. They struggle sometimes editing it (messing up parens)), but usually the first pass is pretty good.

I've been on an LLM kick the past few months, and two of my favorite AI-coded (mostly) projects are, interestingly, REPL-focused. icl (https://github.com/atgreen/icl) is a TUI and browser-based front end for your CL REPL designed to make REPL programming for humans more fun, whether you use it stand-alone, or as an Emacs companion. Even more fun is whistler (https://github.com/atgreen/whistler), which allows you to write/compile/load eBPF code in lisp right from your REPL. In this case, the AI wrote the highly optimizing SSA-based compiler from scratch, and it is competitive against (and sometimes beating) clang -O2. I mean... I say the AI wrote it... but I had to tell it what I wanted in some detail. I start every project by generating a PRD, and then having multiple AIs review that until we all agree that it makes sense, is complete enough, and is the right approach to whatever I'm doing.

Thank you!

The whistler code you inline with your common lisp is an s-expression based DSL. So you can use common lisp macros, but those macros are generating something that will look familiar to CL devs but is restricted based on the eBPF archictecture and validator requirements. eg. it only supports bounded `dotimes`, some basic progn/let/when/if/cond/eq/setf/incf/decf and math, and a simple array iterator. No lists, loops, tagbody/go, conditions, etc, etc. There's a manual in the docs directory.

They are much better these days.

256k is just's just a placeholder for now. The default will get reduced as we get more experience with the draft implementation. The proposal isn't complete yet.

I haven't really looked into it, but I'm hopeful it can be made to work.

No. I have yet to propose the patches formally. The SBCL maintainers are reviewing the high-level proposal (on my blog) first. You can try the implementation, however. There's a pointer to the repo/branch on my blog. I need to build a proper benchmarking framework and publish some real numbers that people can reproduce before I am confident enough to submit the patches for review.

Let me know if you try it out. I would love some feedback (via github)

This very website that you are using right now, Hacker News, runs on sbcl.

I'm the author. https://atgreen.github.io/repl-yell/posts/sbcl-fibers/

I set out to create a better text repl experience for Common Lisp couple of months ago, but was inspired by the pharo interface and built something much larger: https://atgreen.github.io/icl/ I use it all the time.

It's like Dr Ian Malcolm says.. "Your developers were so preoccupied with whether or not they could, they didn't stop to think if they should" - Jurrasic Park (almost)

Here's something I wrote about this work: https://atgreen.github.io/repl-yell/posts/cl-kawa/

If you are interested in this, you might also be interested to learn that I also got clojure running on SBCL via OpenLDK. See https://github.com/atgreen/cl-clojure.

Regarding LLM-usage, the bulk of OpenLDK was written without the use of LLMs. But recently I let Claude loose on the code to fix a few remaining problems blocking kawa. Claude also upleveled the Java support from Java 8 to Java 21.

I wrote a couple of blog entries related to this work that might be of interest. One was around how I had to use the MOP to optimize method dispatch in CLOS for clojure: https://atgreen.github.io/repl-yell/posts/clos-mop-dispatch/

I managed to get a clojure REPL running on Common Lisp (SBCL) via OpenLDK, my Java JIT compiler and runtime on Common Lisp. This blog post gives an overview on how I manipulated CLOS's method dispatch to improve performance for single-dispatch methods calls.