* * *

From what I see in your example, you invoke the form again. In Common Lisp you don't need that. You can stay in a computation and fix&resume from within.

> Hmm, what advantage does Lisp offer here over Python?

It does have a clear advantage if instead of

    (+ 1 (foo 20))

we were doing

    (+ (long-computation-answering-the-ultimate-question-of-life-the-universe-and-everything)
       (foo 20))

(Reminder: we're dicussing Common Lisp here.)

I have been playing again with CL recently and am doing some trivial web-scraping of an old internet forum. I don't use a REPL directly, but just have a bunch of code snippets in a lisp file that I tell my editor to evaluate (similar to Jupyter?). I haven't bothered doing any exception (condition) handling, and so this morning I found this in a new window:

   Condition USOCKET:TIMEOUT-ERROR was signalled.
      [Condition of type USOCKET:TIMEOUT-ERROR]

   Restarts:
    0: [RETRY-REQUEST] Retry the same request.
    1: [RETRY-INSECURE] Retry the same request without checking for SSL certificate validity.
    2: [RETRY] Retry SLIME interactive evaluation request.
    3: [*ABORT] Return to SLIME's top level.
    4: [ABORT] abort thread (#<THREAD tid=17291 "worker" RUNNING {1001088003}>)

plus the backtrace. This is in a loop that's already crawled a load of webpages and has some accumulated some state. I don't want a full redo (2), so I just press 0. The request succeeds this time and it continues as if nothing happened.

You are not fixing the issue in the dynamic context of a running program. Doesn't matter in this trivial example but is very noticeable when you have a loaded DB cache and a few hundred active network connections.

> Hmm, what advantage does Lisp offer here over Python?

In lisp, I never edit code at the REPL, yet the REPL is what enables me to edit code anywhere. I edit the source files and have my editor eval the changes I made in the source. This gets me the benefit that should my changes work, I don't have to retype them to get them into version control. This works because the Lisp REPL is designed to be able to switch into any existing package, apply code there, and also switch back to the CL-USER package after. My editor uses the same mechanism and only has to inject a single prefix (`in-package :xyz`) before it pastes the code I've selected for eval.

In Python, editing a method in a class inside some module (i.e., not toplevel) is less easy. At least, I haven't found any editor support for it. What I did find is the common advice to just reload the whole module/file.

Okay, so let's reload the whole module, then? Well, Python isn't really built for frequent module reloads and that can sometimes bite. In Common Lisp, the assumption that any code may be re-eval-ed is built in. For example, there's two ways of declaring a global value in CL: defvar and defparameter. The latter is simply an assignment of a value to a variable in the global scope, but the former is special. By default, `defvar` defines a variable only if it's not already defined. So that a CL source file may be loaded and reloaded any number of times without resetting a global variable.

Then there's classes. Oh my. Common Lisp has the most powerful (in terms of flexibility) OO system I know of. Not only can you redefine functions and methods, you can even redefine classes dynamically. Adding a property to a class adds that property to all existing objects of that class. Removing a property from a class removes it from all existing objects of that class. This feature is no longer CL-exclusive, but it is sufficient to offer a massive advantage over Python. I don't need to talk about method combinations, multi-methods and the many other cool features of the Common Lisp Object System here.

Then there's the debugging system. In Python, when an exception is thrown, it immediately unwinds the stack all the way up until it is first caught. So not only do you need to know beforehand where to catch what exception, if you get it wrong you cannot inspect the site of the error. In CL, a condition ("exception") does not unwind the stack until a restart is chosen. Not when it is caught, but rather when — after being caught — a resolution mechanism has been chosen. This allows interactive debugging (another cool CL feature) to inspect the stack frames at (and above) the site of error, redefine whatever code needs to be corrected, all before the error is allowed to unwind and destroy the stack. You still need to set-up handlers (and restarts) before the error happens, but you can be absolutely wildly lax and use catch-all handlers anywhere on the stack and restarts that take absolutely anything (even functions) at debug-time so you don't really need to be prescient with your error handling code unlike in Python.

I'm sure there's more, but I think this is pretty sufficient.

You got a lot of correct but verbose responses. Put in layman's terms you had to run 1 + foo(20) again. If 1 + foo(20) were replaced by a complex and long winded function you would have lost all of that state and needed to run it all again. What if 1 + foo(20) had to read several TB of data in a distributed manner. You would have to do that all again.

There are ways around this and of course you could probably develop your own crash loop system in python but in lisp you simply continue where it failed. It's already there.

You mention doing things in Jupyter and ETLs which are often long running. This could be hugely beneficial to you.

You're losing all the state since you're not being dropped in the closure where the error happened but in the end of the program.

To see the difference use some function counter() instead of 1 in the example.

The advantage is that Python has just diagnosed the error and aborted the whole thing back to the top level, whereas in the Common Lisp, the entire context where the error happened is still standing. There are things you can do like interactive replace a bad value with a good value and re-try the failed computation.

In lispm's example, the problem is that there is no foo function, so (foo 20) cannot be evaluated. You have various choices at the debugger prompt; you can specify a different function, to which the same arguments will be applied. Or just specify a value to be used in place of the nonworking function call.

Being able to fix and re-try a failed expression could be valuable if you have a large running system with hundreds of megabytes or even gigabytes of data in the image, which took a long time to get to that state.

>Note that we are not in some debug mode, to get this functionality.

Jesus Christ I swear it's like you ascribe mysterious powers to the parens. Do you think the parens give you the ability to travel through time or reverse the pc or what? Okay it's not in a debug mode but it's in a "debug mode". Like seriously tell me how you think this works if it's not effectively catching/trapping some sigkill or something that's the equivalent thereof?

I have never in my life met this kind of intransigence on just manifestly obvious things.

Common Lisp programs run by default in a way that calls to undefined functions are detected.

Here the Lisp simply tries to look up the function object from the symbol. There is no function, so it signals a condition (aka exception). The default exception handler gets called (without unwinding the stack). This handler prints the restarts and calls another REPL. I define the function -> the symbol now has a function definition. We then resume and Lisp tries again to get the function definition. The computation continues where we were.

That's the DEFAULT behavior you'll find in Common Lisp implementations.

>Common Lisp programs run by default in a way that calls to undefined functions are detected.

Cool so what you're telling me is that by default every single function call incurs the unavoidable overhead of indirecting through some lookup for a function bound to a symbol. And you're proud of this?

Compared to Python, Common Lisp hardly has any performance issues.

Okay well when pytorch, tensorflow, pandas, Django, flask, numpy, networks, script, xgboost, matplotlib, spacy, scrapy, selenium get ported to lisp, I'll consider switching (only consider though since the are probably at least another 20 python python packages that I couldn't do my job without).

That is a typical consumerism: "Give me everything ready to use and then I'll use it."

How about bringing some value to the community?

For the sake of anyone reading this thread who isn't in the know: many of these libraries are really written in C/C++ and have Python bindings.

i said ported not implemented; the likelihood that any of those libraries sprout lisp bindings is about as likely as them being rewritten in lisp. so it's the same thing and the point is clear: i don't care about some zany runtime feature, i care about the ecosystem.

Stop moving the goalposts: your answer to a commenter who stated that Common Lisp was faster than Python (a fact) was a list of packages, many of which are (1) not even written in Python and (2) some of them actually do have Common Lisp bindings.

I thought you know Lisp? Now you are surprised that Lisp often looks up functions via symbols -> aka "late binding"? How can that be? That's one of the basic Lisp features.

Next you can find out what optimizing compilers do to avoid it, where possible or where wanted.

At no point in time did I claim to know lisp well. I stated my familiarity at the outset. But what you all did was claim to know a lot about every other interpreted runtime without a grain of salt.

>Next you can find out what optimizing compilers do to avoid it, where possible or where wanted.

But compilers I am an expert in and what you're implying is impossible - either you have dynamic linkage, which means symbol resolution is deferred until call (and possibly guarded) or you have the equivalent of RTLD_NOW ie early/eager binding. There is no "optimization" possible here because the symbol is not Schrodinger's cat - it is either resolved statically or at runtime - prefetching symbols with some lookahead or cabinet is the same thing as resolving at calltime/runtime because you still need a guard.

> But compilers I am an expert in and what you're implying is impossible

> it is either resolved statically or at runtime

Just tell Lisp which calls to statically resolve, inline, optimize. Overwrite the global default.

  (defun foo (a)
    (declare (inline +)
             (optimize (speed 3))
             (type (integer 0 100) a))
    (* 10 (+ 3 a)))

Above tells Lisp to inline the + function, optimize for speed and declares the type of a to be an integer in the range 0 to 100.

  * (disassemble #'foo)
  ; disassembly for FOO
  ; Size: 32 bytes. Origin: #x7006DC8544  ; FOO
  ; 44:       40190091         ADD NL0, R0, #6
  ; 48:       5C0180D2         MOVZ TMP, #10
  ; 4C:       0A7C1C9B         MUL R0, NL0, TMP
  ; 50:       FB031AAA         MOV CSP, CFP
  ; 54:       5A7B40A9         LDP CFP, LR, [CFP]
  ; 58:       BF0300F1         CMP NULL, #0
  ; 5C:       C0035FD6         RET
  ; 60:       E00120D4         BRK #15    ; Invalid argument count trap

As you can see in the machine code, Lisp then uses the native machine code ADD and MUL instructions.

What you're missing is that, unlike any other commonly used language runtime, compilation in CL is not all-or-nothing, nor is it left solely to the runtime to decide which to use. A CL program can very well have a mix of interpreted functions and compiled functions, and use late or eager binding based on that. This is mostly up to the programmer to decide, by using declarations to control how, when, and if compilation should happen.

To be fair, any dynamic language with a JIT will mix interpreted and compiled functions, and will probably claim as a strength not leaving to the programmer the problem of which to compile.

Opinions may vary on that point.

It should also be noted that by spec symbols in the system package (like + and such) should not be redefined. This offers “unspecified” behavior and lets the system make optimizations out of the box.

Outside of that you can selectively optimize definitions to empower the system to make better decisions at the cost of runtime protection or dynamism. However these are all compiler specific.

* * *

Do you think Python or Ruby or PHP are any different? And yet, not one of them actually chose to use this in a sane way, where a simple lookup error doesn't have to crash the whole program.

That depends. The Common Lisp standard says nothing on the subject. CMUCL[1] and its descendent SBCL[2] do something clever called local call. It's not terribly difficult to optimize hot spots in your code to use local call. Outside of the bottlenecks, the full call overhead isn't significant for the overwhelming majority of cases. It's not like full call is any more expensive than a vtable lookup anyhow.

[1] https://cmucl.org/downloads/doc/cmu-user-2010-05-03/compiler...

[2] https://www.sbcl.org/manual/#Miscellaneous-Efficiency-Issues

* * *

> If f and g are in the same file, where g calls f, then implementations are not required to allow f and go to be separately redefinable. So that is to say, if f is redefined only, the existing g may keep calling the old f. The intent is that redefinition has the granularity of compiled files: if a new version of the entire compiled file is loaded, then f and g get redefined together and all is cool.

This isn't the default behaviour though, right?