News for hackers. Stuff that matters.
Re:This Old Lisp by eschaton on Friday October 19, 2018 @01:44AM (#18254451)
My main beef with treating the Common Lisp spec as a sacred text is simple:

1. There were things at the time that were not quite standardizable that are now. Examples: Some degree of concurrency, system definition, the CLOS meta-object protocol, networking.

2. There are things that were created or became important in the meantime that should be standardizable now: Unicode and localization, concurrency building blocks like threads and locks and queues and atomic operations, cryptography, compression, higher level networking (URLs, HTTP, etc.).

3. There are just bugs and holes and incompleteness in the standard due mostly to compromise from the then current vendors. For example, not everything fits the sequence protocols, the language isn’t CLOS all the way down, streams, etc.

All this argues that there’s room for continued progress. The only good argument I’ve heard for why it hasn’t been pursued is “Who’d fund it?”

Re:This Old Lisp by mark_l_watson on Friday October 19, 2018 @10:57PM (#18261473)
I don’t particularly mind that Clozure CL, SBCL, Franz, Armed Bear CL, LispWorks, etc. all have different solutions for some things not in the spec.

When I retire from full time work next year, I have planned on just using one language, probably either Common Lisp or Racket, for research and recreational programming. The fact that the CL standard is crufty doesn’t really put me off. Racket does handle many of the issues you mention nicely but there is something nice about using a programming language that I have been using for 36 years.

That said, if DARPA and/or a large consortium of corporations decided to drop $$$ to re-standardize and improve the language then that would be very cool. But, I don’t need that to happen.

Re:This Old Lisp by blue1 on Friday October 19, 2018 @02:27AM (#18254585)
I think it's evident now that a purely community-based effort to advance CL cannot realistically happen; even the CDR[1] is unmantained, and in the course of the last years many knowledgeable people have been progressively leaving the community, CL, or both.

I am wondering how the "other" languages obtain funding for such things. For example, Julia raised $4M one year ago. Or should we acknowledge that no one cares about Lisp anymore? :(

[1] https://common-lisp.net/project/cdr/

Re:This Old Lisp by vindarel on Friday October 19, 2018 @07:06AM (#18255768)
Just to counterbalance and give some hope: projects like CLASP (CL on LLVM) got university funds to development it (and the ecosystem on the way: LSP library,…).

https://github.com/clasp-developers/clasp

CL is still adopted by universities (for bioinformatics it seems), or innovative firms (Emotiq (blockchain), Rigetti quantum computing,…), and is still used by big ones (https://github.com/azzamsa/awesome-lisp-companies), so why not see fundings in the future…

Re:This Old Lisp by sdegutis on Friday October 19, 2018 @10:42AM (#18257017)
What language have they mostly been moving to?
Re:This Old Lisp by dwc on Friday October 19, 2018 @09:31AM (#18256568)
> the language isn’t CLOS all the way down

I'm curious what you mean by this, why it's needed or would be a good thing, etc. As a multi-paradigm language, I'm not seeing why CL should have a particular paradigm "all the way down".

Re:This Old Lisp by mikelevins on Friday October 19, 2018 @11:32AM (#18257409)
It's good because it offers the opportunity to simplify and rationalize the type system and associated protocols without losing features.

In the early 1990s I worked on an experimental Newton OS written in Dylan. At that time, Dylan was still called "Ralph," and it was basically an implementation of Scheme in which all datatypes were CLOS classes. It was "CLOS all the way down."

Ralph offered substantially the same affordances and conveniences as Common Lisp, but with a simpler and more coherent set of APIs. Ralph was easier to learn and remember, and easier to extend.

To illustrate why, consider finite maps. The idea of a finite map is a convenient abstraction with a well-defined API. Common Lisp offers a couple of convenient ways to represent finite maps, and it's easy to build new representations of them, but there's no generic API for finite maps. Instead, each representation has its own distinct API that has nothing particularly to do with anything else.

By contrast, Ralph had a single well-defined API that worked with any representation of finite maps, whether built-in or user defined.

The upshot is a library of datatypes that is just as rich as Common Lisp's, but with a simpler and more coherent set of APIs, and an easy standard way to extend them with user-defined types that also support the same APIs.

There are signs in the Common Lisp standard that people were already thinking in that direction when the standard was defined. See the sequence functions, for example. Ralph, designed by Common Lisp and Smalltalk implementors, carried that thinking to its logical conclusion, and the result was something like a tidier Common Lisp.

Twenty-eight years later, Ralph is still my favorite of all the languages I've used for serious work. Its development system, Leibniz, remains my favorite development system. My favorite current tools are Common Lisp systems, but that's because I can't have Ralph and Leibniz anymore.

Re:This Old Lisp by mindcrime on Friday October 19, 2018 @11:42AM (#18257502)
My favorite current tools are Common Lisp systems, but that's because I can't have Ralph and Leibniz anymore.

You said below that you don't find modern day Dylan to be as valuable. I don't know much about Dylan, either the pre-1992 version or the newer version(s), but I'm curious if you would elaborate on why the older Dylan was so much superior to modern Dylan in your view?

Re:This Old Lisp by mikelevins on Friday October 19, 2018 @12:28PM (#18257839)
Because modern Dylan is not an old-fashioned Lisp.

I prefer working the old-fashioned Lisp way. I start my Lisp and tell it, an expression at a time, how to be the app I want. Modern Dylan doesn't work like that. It's much more a batch-compiled affair, where you write a lot of definitions and compile them all at once to yield an artifact.

Modern Dylan does not have a Lisp-style repl that you can use to gradually build up your app interactively, teaching the runtime new tricks, and incrementally querying it to examine what you've built--as I did when working on the Dylan Newton.

For a while, Bruce Michener and I discussed what it would take to restore that kind of support to OpenDylan, but in the end I concluded it was an impractical amount of work.

Re:This Old Lisp by mindcrime on Friday October 19, 2018 @12:56PM (#18258068)
Gotcha. I've never really used OpenDylan, but I've had it on my "list of things to learn one day" for a while, so just curious about your take on that. I didn't realize that old Dylan had a lisp style REPL and that new Dylan doesn't.
Re:This Old Lisp by Immortalin on Friday October 19, 2018 @12:25PM (#18257814)
What's your opinion on Julia given its Dylan heritage?
Re:This Old Lisp by mikelevins on Friday October 19, 2018 @12:31PM (#18257873)
Julia is almost good enough for me to use, but not quite. I'd prefer an s-expression syntax, but that's not a deal killer for me. I like some other languages that don't have s-expression syntaxes, though I miss the easy text manipulation that s-expressions support.

I would want a convenient way to deliver a self-contained executable. If there's a simple way to do that with Julia, I don't know about it. I look for it periodically, but haven't found it. If it's there and I've simply overlooked it, then I might actually start using it regularly.

I have a few other nits, but they're just nits. On the whole, I think Julia's pretty nice.

Re:This Old Lisp by mikelevins on Friday October 19, 2018 @10:08PM (#18261321)
One thing I forgot to mention is that I don't know whether Julia handles dynamic redefinitions gracefully.

What I mean is, for example, if I evaluate a new definition of an existing class, what happens to all the existing instances of that class? In Common Lisp, the old instances are now instances of the new class, and there is a runtime facility, defined in the language standard, for updating existing instances to ensure that they conform to the new definition.

If a language lacks facilities like that, then it's hard to work the way I prefer to work.

I guess I sort of expect that Julia will not have graceful support for redefinitions, because, generally speaking, the only people who even think of that feature are people who are intimately familiar with old-fashioned Lisp and Smalltalk systems, and they're sort of thin on the ground.

But maybe I'll be pleasantly surprised.

Re:This Old Lisp by elcritch on Saturday October 20, 2018 @12:10AM (#18261730)
Julia doesn’t have classes. It relies on multi-methods primarily. Which for scientific computing is a much better fit, IMHO. That being said it’s possible to redefine pretty much any operator, including built in ones at the repl.
Re:This Old Lisp by mikelevins on Saturday October 20, 2018 @12:52AM (#18261828)
Your reply is a bit confusing, because multimethods are not an alternative to classes. Common Lisp and Dylan, for instance, offer both multimethods and classes.

Regardless, Julia does offer user-defined composite types. Can I redefine a composite type without halting the program in which it's being used? If so, what becomes of existing instances of the type?

If the answer to the first question is "yes," and if the answer to the second one is "the language runtime arranges for the existing instances to be updated to be instances of the redefined type," then Julia offers the kind of support for redefinition that I am accustomed to in Common Lisp. If not, then it doesn't.

EDIT: I dug around and answered my own question: Julia doesn't support redefining structs in the repl.

There's a project in progress (Tim Holy's Revise.jl) to add support for redefining functions in a session, and that project contains some discussion of how they might approach redefining structs.

Of course, the existence of the project and those comments implies that Julia does not currently support such redefinitions, and that answers my questions.

I did notice from the comments on some issues that those folks are aware that supporting redefinition of structs in the repl implies that existing instances may become orphans when their types are redefined, and there's some discussion of what to do about it. Common Lisp's solution--updating the existing instances to conform to the new definition--does not seem to have occurred to anyone.

That's not a big surprise. Why would such a feature occur to you unless you were consciously designing a system for building programs by modifying them as they run? Of course, that's exactly what old-fashioned Lisp and Smalltalk systems are designed for, but most people don't get much exposure to that style of programming.

I always end up missing those features when I don't have them, though, which is one reason I always end up going back to Common Lisp.

Re:This Old Lisp by elcritch on Saturday October 20, 2018 @02:17AM (#18262013)
True, classes and multimethods aren’t exactly interchangeable. It’s just that I haven’t found classes useful (at least as done in Java/Python/C++ objects) as compared to the combination of multimethods and type specialization. At least in the context of scientific computing.

Does CL use virtual tables to implement CLOS? Always been curious about that. It seems CL must keep the state associated with redefined objects. How do you handle new fields and filling in values with CLOS?

It does appear you can’t redefine structs in the repl. Forgot about that point, though as you point out there doesn’t appear to be anything fundamental to prevent that from being changed in the future. I haven’t used Julia day-to-day much for a while, but hopefully the newer generation tools will add in the “old” features from CL and similar.

Have you ever tried CLASP?

Re:This Old Lisp by mikelevins on Saturday October 20, 2018 @03:51AM (#18262268)
CLOS classes are logically equivalent to structs. In fact, in some implementations, they are exactly the same. They are therefore useful in exactly the same ways and the same circumstances that structs are useful.

Maybe what you don't find useful is inheritance. I can see that. I'm not heavily invested in inheritance myself, though it can be useful in cases where you want a bunch of structs with some shared structure, or in cases where you want multimethods that share some behavior across a bunch of related types.

The terminology "virtual table" is commonly used with C++ and other languages that associate methods with classes. Each class in such languages has a hidden member that contains a pointer to a virtual method table used for method dispatch.

In CLOS, methods are associated with generic functions, not with classes, and are dispatched on any number of arguments. The standard specifies how generic functions and methods behave, but does not specify how they are to be represented, so the representation is an implementation-specific detail.

A naive toy representation might be a table associated with each generic function that maps sequences of types to methods. When the function is applied, Lisp computes the values and types of the arguments and finds the appropriate method for those types. I'm sure you can imagine the sorts of optimizations implementations apply to speed things up, including compiling monomorphic generic functions to simple function calls.

This is a bit of an oversimplification, because CLOS also provides a bunch of ways to control and customize how dispatch works--CLOS is less an object system than it is a system for building object systems.

When you redefine a class, CLOS automatically calls MAKE-INSTANCES-OBSOLETE, which arranges for all existing instances to be marked obsolete (it's up to the implementation to determine exactly what that means). When control touches an obsolete instance, the runtime calls UPDATE-INSTANCE-FOR-REDEFINED-CLASS with the instance, a list of added slots, a list of discarded slots, and a property list mapping the names of discarded slots to the values they had when they were discarded. If you've specialized UPDATE-INSTANCE-FOR-REDEFINED-CLASS for the case in question, the instance is reinitialized according to your specialized method, and things proceed as if it had the new type definition when it was instantiated.

If you haven't specialized UPDATE-INSTANCE-FOR-REDEFINED-CLASS then you'll end up in a breakloop. A breakloop is a repl session with read and write access to the call stack and the variable environment. The assumption is that you'll inspect the stack and environment, decide what UPDATE-INSTANCE-FOR-REDEFINED-CLASS needs to do, write that code, then invoke a restart that causes the halted function to resume execution as if your new definition had existed when it was originally called.

Again, the language is designed with the assumption that writing a program by modifying it while it runs is standard operating procedure. That being the case, the obvious thing to do when there isn't a relevant definition for UPDATE-INSTANCE-FOR-REDEFINED-CLASS is to offer you the chance to create one, and resume execution from there once you've created it.

I've examined CLASP a bunch of times. I keep meaning to mess with it, but I haven't yet.

Re:This Old Lisp by fasquoika on Friday October 19, 2018 @07:34PM (#18260849)
>I would want a convenient way to deliver a self-contained executable.

It's a bit rough around the edges, but it does exist: https://github.com/JuliaLang/PackageCompiler.jl

Re:This Old Lisp by mikelevins on Friday October 19, 2018 @09:12PM (#18261144)
Excellent; thank you! I'll take a look.
Re:This Old Lisp by throwaway487548 on Friday October 19, 2018 @11:04AM (#18257162)
Uniformity, which is a really good thing. Surely you could say (class-of 3) or (class-of nil) or (class-of '(1 2 3)) but technically these values are not objects, like it is in, say, Scala, which is a real-world example of how good it is to have a uniform language (everything is an expression, every value is an instance of a class, and therefore everything is uniformly high-order, uniformly typed (unlike Java with distinctions of so-called primitive types) etc, etc.
Re:This Old Lisp by dwc on Friday October 19, 2018 @04:16PM (#18259685)
Uniformity through imposing one paradigm on everything isn't attractive at all to me, especially for a paradigm I have no interest in using and avoid when I practically can.
Re:This Old Lisp by junke on Friday October 19, 2018 @11:42AM (#18257508)
Every value in CL is an instance of class. Some of those classes are built-in classes, which are restricted for performance reason. You do not inherit from Int in Scala either, since it is marked as "final", as far as I know.
Re:This Old Lisp by throwaway487548 on Friday October 19, 2018 @12:05PM (#18257687)
In Scala an integer has methods, like everything else, unlike it is in Java and C++, and this is the point and the big deal.

    3 + 2 is actually 3.+(2) which is the right thing.
Re:This Old Lisp by kbp on Friday October 19, 2018 @07:21PM (#18260781)
You can specialise generic functions on built-in classes in standard CL. Lisp methods are specialisations of generic functions; they don't belong to a class in the way methods do in eg C++. The issue you're talking about is that not all functions are generic functions in Common Lisp, and you can't specialise ordinary functions.

There's nothing stopping you from doing

    (defmethod add ((x number) (y number)) (+ x y))
    (defmethod add ((x string) (y string)) (concatenate 'string x y))
or whatever (multiple dispatch, too), and you could even call it + instead of ADD if you wanted (but not COMMON-LISP:+, so other code would continue to work; your packages could import your + instead of the standard one).
Re:This Old Lisp by junke on Friday October 19, 2018 @12:20PM (#18257776)
"+" is a function, what makes it "right" to be a method?
Re:This Old Lisp by pfdietz on Monday October 22, 2018 @09:41PM (#18280149)
You mean, what makes it right to be a generic function that has methods?

First, + does dynamic dispatch based on the types of its arguments. It does different things when adding fixnums, vs. integers, vs. rationals, and so on, as well as a default method that signals a type error (in safe code). So it has methods, even if they aren't necessarily implemented as standard methods (but they could be).

Secondly, a user might want to make + work on other, user-defined classes (for example, if he user wanted it to work on a class representing quaterions). To make that work, the user would have to be able to add methods for those classes. One can imagine many CL builtins being implemented as generic functions to which users could add methods. This would be consistent with the standard.

Re:This Old Lisp by jhbadger on Saturday October 20, 2018 @07:29AM (#18262769)
a function is just a method that returns a value. Why make a special case for it? Of course you can go the other direction and allow functions to return nothing (or a representation of nothing like nil). That's fine too.
Re:This Old Lisp by e12e on Friday October 19, 2018 @11:22AM (#18257314)
But would such a common lisp be better than something like Dylan?
Re:This Old Lisp by mikelevins on Friday October 19, 2018 @11:33AM (#18257419)
It would not be better than circa-1992 Dylan. It would be better than present-day Dylan, though.

My opinion only, of course.