A cool example of this was NASA's Deep Space 1 probe. The control system was written in Lisp. There was a race condition in the code that wasn't detected during ground testing. NASA software engineers were able to remotely debug and patch the code while it was still running.

ti.arc.nasa.gov/m/pub-archive/176h/0176%20(Havelund).pdf

Imagine running a REPL with 45 minutes of light-speed delay in the loop. I love to tell this anecdote

Macros are fully expanded before the rest of the code is executed. Trying to argue that Lisp doesn't have a distinction between compile time and run time because you usually have the compiler (or at least interpreter) available at runtime strikes me as just playing semantics games.

Do you have a citation for this? I was under the impression it was entirely possible for lisp macros to be expanded at runtime. For example, you could have data that is passed to a function and then used as code (because code and data are the same thing). The system doesn't know it's going to be used as code until it is, so the macros can't be expanded until then.

If Lisp uses an interpreter (which runs Lisp source code), you may see macro expansions at runtime.

For compiled code, there usually will be no macro expansion at runtime. Lisp will not macro expand code in compiled code, since all macros need to be expandable at compile-time.

But: if you explicitly want it, you can generate code at runtime and then you need to call EVAL or COMPILE. Then macro expansions might happen.

I think this is why it's non-trivial to build a distributable exe of the program, right? The exe would need a lisp interpreter/compiler bundled with it, because at runtime it might generate some code and run it!

Except that it's actually very trivial to make such an exe.

From [0]:

"Most Common Lisp implementations support creating native-format executable files. The typical route involves loading all of the program's code into the Lisp environment, dumping the current state of the Lisp process into a disk image, and identifying a start function (ie main).

Since the disk image contains the entire Common Lisp system (compiler, debugger, documentation strings, etc.), the executable will be typically larger than a compiled C program (even if you statically link the C code).

Another form of disk image requires a copy of the particular Lisp implementation to be installed on the recipient's machine. This is no different in principle from a C application requiring platform-specific dynamic libraries to run. This complicates deployment - the disk image is tied to a particular implementation version, and you typically end up having to include the Lisp implementation executable anyway.

Due to platform limitations, some Lisp implementations can't produce native executables (for example, ABCL can only make JAR files). Other implementations are only able to dump disk images that require a separate copy of the implementation executable. An alternative to executables is available on Unix platforms in the form of shell scripting."

If it needs 4 wordy paragraphs and you still don't know how to do it, I am going to scribe that up as non-trivial.

[0] https://www.cliki.net/Creating%20Executables

That's because different implementations provide different ways to generate executables. Some are simple, some are complex.

For SBCL one needs to call a function, which creates the executable image.

http://www.sbcl.org/manual/index.html#Function-sb_002dext-sa...

There are more complex machineries for application generation, though.

You don't need those 4 paragraphs.

I have this alias in my .zshrc:

    clc() { sbcl --no-userinit --load $1 --eval "(sb-ext:save-lisp-and-die \"$(sansext $1)\" :toplevel 'main :executable t)" }

Sansext is a custom function that strips the file extension from a filename, but you don't need that; you can read output filename from $2, or whatever else.

Nice. I think it is important for proponents of languages to answer FAQs like this so you aren't left in a corner when you want do so something (i.e build & distribute) that is standard in most languages.

Every reference on Lisp macros says that they are executed at compile-time.

From http://www.paulgraham.com/icad.html: There is no real distinction between read-time, compile-time, and runtime. You can compile or run code while reading, read or run code while compiling, and read or compile code at runtime.

That is from one of the founders of YCombinator, the author of http://www.paulgraham.com/onlisptext.html and http://www.paulgraham.com/acl.html.

There is a distinction. I can compile code while reading, but when I'm compiling, then I'm in compile-time.

In a compiled implementation, code needs to be compiled before it gets run. When I want it to be compiled, I have to call functions like COMPILE or COMPILE-FILE.

That's the kind of misleading text I was talking about in my first comment.

There is a conceptual distinction between read time, compile time, and run time, but it's only conceptual: all these things happen in a single process in an interleaved manner. When you type an expression into the REPL, first it gets read, then it gets compiled to some extent that depends on the implementation, then it gets run. Some implementations run every expression you type through the full native-code compiler, but others just interpret the expression, expanding any macros as they encounter them. In the latter case, it's switching back and forth between "run time" and "compile time" potentially many times during the interpretation of a single expression.

It is certainly true that macros are a different kind of code from most ordinary code, because what they're doing is different. But importantly, they are written in the same Turing-complete language. I once wrote a program (a C implementation for Lisp Machines) that allowed the user to interactively execute C expressions. Obviously, it had to parse the C code according to the syntax of that language, but the parser produced list structure that consisted entirely of macro calls. The guts of the compiler were implemented as a collection of macros that translated the output of the parser into Lisp Machine Lisp. These macros did things like accessing and updating a symbol table — far beyond what you could do in a C macro.

I can compile a file to machine code and load it into another process. The file compiler provides the 'compile-time'. The other process then provides 'load-time' and 'run-time'.

Right, you can also do it that way. I should have said "all these things can happen in a single process in an interleaved manner".

If anyone's still reading, I can't resist adding that I was doing type checking by macro expansion — a technique that was the subject of a 2017 POPL paper [0] — in 1983.

[0] https://www.ccs.neu.edu/home/stchang/pubs/ckg-popl2017.pdf

That's so cool! Yet another example of Lisp enabling what other languages get in the way of.

Then today you learn that Paul Graham is a tool.

There are Lisps that improve on the situation. In particular, vau calculi/fexprs are worth examining. http://lambda-the-ultimate.org/node/4093

There are also systems like Forths, where the compiler and interpreter really are the same system, in two different modes of operation. In these systems, there is only runtime; compiletime is a specific style of runtime.

Could you expand on how Forth differs from Lisp? I don't know Forth.

In Forth, the interpreter reads tokens. By default, each token is read and then executed. There is a token that switches the interpreter into "compiling" mode. When in compiling mode, each token is read and then used to instruct the compiler to emit code representing the token. There is a traditional and beautiful interplay [0] between the Forth interpreter and its initial stream of tokens, as the tokens customize the interpreter and compiler by augmenting their behaviors.

The main contrast that I would draw between Forths and Lisps is syntax. Forths don't really have syntax; they have token-parsing streams. Lisps are extremely tree-oriented, but Forths are stack-oriented.

[0] https://github.com/nornagon/jonesforth

Forth has two singleton stacks, and each has a DSL. Sometimes I wonder what Forth would be like with stacks as a first-class type. Maybe actors each with their own stack …