News for hackers. Stuff that matters.
Re:Lisp, Smalltalk, and the power of symmetry (2014) by adamddev1 on Sunday May 29, 2022 @07:06PM (#31553763)
There's something magically expressive about S-expressions and processing them. For a linguistic project I was thinking about how to represent sentence syntax as data and then I came across Chomsky's phrase structure analysis[1] which parses spoken language exactly like S-expressions in LISP! It's even illustrated with parenthesis or syntax trees, just like the old SICP vids explain. The phrase structure even uses recursive definitions so processing and inflecting the content of sentences can be done with recursive functions etc, (I did it in TypeScript) like something following a design exercise out of HTDP/SICP. I think the inherent parallel between human language phrase structure and S-expressions is beautiful.

[1]: This book introduces the phrase structure in a shockingly LISP-like way. https://www.amazon.com/Grammar-Frank-Palmer/dp/B000S5VSAS

https://en.wikipedia.org/wiki/Phrase_structure_rules

Re:Lisp, Smalltalk, and the power of symmetry (2014) by skissane on Sunday May 29, 2022 @08:06PM (#31554141)
> There's something magically expressive about S-expressions and processing them.

A downside I see with S-expressions – you often end up with something like (FOO BAR BAZ), where FOO identifies the type of thing, and BAR and BAZ are parameters/slots/etc – but you have to remember the (sometimes quite arbitrary) order that BAR and BAZ go in.

Some Lisp family languages (such as Common Lisp, Clojure, Scheme with SRFI 88/89) support keywords (aka named parameters), so you can do something like (FOO :bar BAR :baz BAZ), which is considered equivalent to (FOO :baz BAZ :bar BAR). However, this is somewhat of a late development in the Lisp tradition, the status of its adoption is mixed, and (arguably) it is moving away (even if only by a little bit) from pure S-expressions. When people say "S-expressions", they normally aren't thinking of CLOS.

I think JSON's model, in which lists and objects are independent first-class entities, with distinct syntax, has something to say for it. The downside of JSON objects, is their keys are strings, not symbols, and they don't have any kind of "class slot" (some people make up conventions here, for example a reserved property name such as "$class", but those conventions are not uniform or standardised). Oh, and we don't need all those commas. I think, if I was designing a LISP-ish language, I'd have a first-class syntax for objects which used different opening and closing characters – maybe (FOO bar baz) for a list, but [FOO :BAR bar :BAZ baz] for an object.

Re:Lisp, Smalltalk, and the power of symmetry (2014) by nlitened on Monday May 30, 2022 @12:12AM (#31555407)
> I think JSON's model, in which lists and objects are independent first-class entities, with distinct syntax, has something to say for it. > Oh, and we don't need all those commas. I think, if I was designing a LISP-ish language, I'd have a first-class syntax for objects which used different opening and closing characters – maybe (FOO bar baz) for a list, but [FOO :BAR bar :BAZ baz] for an object.

I think Clojure has what you're looking for, and the syntax is great, much more light-weight than JSON (the syntax even has its own name: EDN). 'Objects' are first-class hash maps, your example could look like this `{:class FOO :bar BAR :baz BAZ}`.

Re:Lisp, Smalltalk, and the power of symmetry (2014) by didibus on Monday May 30, 2022 @12:52AM (#31555590)
That's exactly what Clojure did, here's the quote from clojure.org:

> Extends the code-as-data paradigm to maps and vectors

Basically think of it as a better JSON with non of the issues you brought up.

See: https://github.com/edn-format/edn

And example:

> #myapp/Person {:first "Fred" :last "Mertz"}

Re:Lisp, Smalltalk, and the power of symmetry (2014) by JonChesterfield on Monday May 30, 2022 @05:30AM (#31557103)
The keyword vs symbol distinction still seems weird to me. :first vs 'first. {} delimited sequence of even length is a nice complement to () delimited sequence of arbitrary length though.
Re:Lisp, Smalltalk, and the power of symmetry (2014) by nlitened on Wednesday June 01, 2022 @03:41AM (#31579181)
I think of it this way: keywords are like symbols, but they survive serialization/deserialization and macroexpansion intact, so you don't need to be extra careful with quoting to prevent automatic evaluation of the symbol to some other value. And since keywords are used in code very often as keys and enums, it reduces cognitive load immensely.
Re:Lisp, Smalltalk, and the power of symmetry (2014) by eadmund on Sunday May 29, 2022 @10:01PM (#31554813)
A flaw with the JSON model is that there is no inherent sorting to the order of pairs. This has problems for canonicalisation, but also for shadowing.

A Lisp can use an alist to represent a set of key-value pairs, and is easy enough to shadow values there:

    (defvar original '((a . 1) (b . 2)))
    → ((A . 1) (B . 2))
    (assoc 'b foo)
    → (B . 2)
    (defvar bar `((b . 3) ,@foo))
    → ((B . 3) (A . 1) (B . 2))
    (assoc 'b bar)
    → (B . 3)
This makes it easy to implement things like inheritance or trees of lexical environments.

I sometimes wish that Lisp had better ergonomics for hash tables, but normally I am pretty happy that folks don't reach for them first. Frankly, I think positionality is normally good. Can you image how terrible it would be to program a language in which addition is represented as:

   add(operand: 1, operand: 2)
Re:Lisp, Smalltalk, and the power of symmetry (2014) by renox on Monday May 30, 2022 @04:26AM (#31556718)
Positionality is good only when arguments are interchangeable/commutative like addition or when the types are different which ensure that any mistakes in the order will be detected by the compiler. In all the other case positionality is bad: see memset for example.
Re:Lisp, Smalltalk, and the power of symmetry (2014) by adamddev1 on Sunday May 29, 2022 @08:57PM (#31554463)
Yes definitely a valid point. For my case I used TypeScript and made a typed array of objects (with recursive definitions to arrays of the same) that cold be arranged and processed like an S-Expression. This typing and having named elements in the objects really hit the sweet spot for me while still being able to process the list/array in a functional/recursive way.
Re:Lisp, Smalltalk, and the power of symmetry (2014) by dreamcompiler on Monday May 30, 2022 @12:39AM (#31555537)
Another flaw in JSON is that there's no built-in way to connote referential integrity. Which is another way of saying that JSON can represent trees but not arbitrary graphs.

Here's a circular list for example in Common Lisp:

#1='(foo bar baz #1#)

[Printing that is a good way to overflow your stack]

More usefully, here is an employee who is an acting supervisor of himself:

'(:employee #1="Frank Smith" :supervisor #1#)

There are ways around this problem in JSON but it requires a bespoke parser. In Common Lisp it's built in to the reader.

Re:Lisp, Smalltalk, and the power of symmetry (2014) by skissane on Monday May 30, 2022 @01:14AM (#31555681)
Self-reference causes a lot of issues. Sometimes I wonder if it should just be prohibited, forcing everything to be a directed acyclic graph.

If a language has only immutable data structures, it is easy to make it so that self-reference becomes impossible, just by making sure your primitives for constructing values don't permit cycles to be constructed.

Even if you allow mutable data structures, it is still possible. For mutable compound heap objects (i.e. heap objects which possibly container pointers to other heap objects), they can have a back-pointer pointing to their parent. If you think about it, you can flesh out some rules about how to manipulate these objects which will make cyclic references impossible. When the rules are violated, rather than failing the operation, you could just add a pointer to a deep copy instead. (Not completely deep, you can still share rather than copy non-compound heap objects, such as strings–sharing non-compound objects can't produce cycles.)

Banning direct self-reference doesn't mean nothing can ever refer to itself, just not directly. So, you couldn't have #1='(employee :name "Frank Smith" :supervisor #1#). But you could have something like '(employee :id 1 :name "Frank Smith" :supervisor (ref :to employee :id 1)). At a physical level there is no self-reference, but at a logical level there still is.

Similarly, one would want to allow cyclic references via a symbol, so a symbol can occur in its own binding. So not completely banning self-reference, just limiting it, in the hope that limiting it will limit the issues it causes. I think, using the strategies I've outlined above, reference counting will work as a garbage collection strategy, without the possibility of memory leaks from cyclic references. (Allowing a symbol's binding to refer to itself will not leak memory, since a bound symbol should still exist even if nobody refers to it; to deallocate it, you have to explicitly unbind it, which will remove the binding, breaking the self-reference and allowing the reference count to fall to zero.)

Re:Lisp, Smalltalk, and the power of symmetry (2014) by JadeNB on Tuesday May 31, 2022 @10:25AM (#31569854)
> If a language has only immutable data structures, it is easy to make it so that self-reference becomes impossible, just by making sure your primitives for constructing values don't permit cycles to be constructed.

Much like the Y combinator comes out somewhat unexpectedly from the pure lambda calculus, my suspicion is that any sufficiently rich data-construction language will allow, intentionally or unintentionally, for some sort of self-reference.

Re:Lisp, Smalltalk, and the power of symmetry (2014) by JonChesterfield on Monday May 30, 2022 @05:36AM (#31557137)
Causes problems with functions though as you lose straightforward recursion. That leaves you with either define creating cycles, or introducing letrec to create cycles, or the dubious beauty of the Y combinator.

The acyclic heap is a great property for most data but it's annoying for representing programs. It might be reasonable to restrict letrec to creating functions, and ensure they're all statically allocated, which would give an acyclic data heap with direct representation of functions.

Re:Lisp, Smalltalk, and the power of symmetry (2014) by skissane on Monday May 30, 2022 @06:27AM (#31557454)
I still think cycles should be allowed via symbols – just not directly. Allowing cycles via symbols tends to be less of a problem: (1) you don't want to garbage collect bound symbols, and once the symbol is unbound the cycle is broken (it would remove the ability to treat environments as first class objects, but maybe the other advantages are worth that cost); (2) printing values generally just prints the symbol name not its binding; (3) very often when you recurse through a data structure, you don't recurse into the binding of symbols contained in it, so that entails a significant reduction in the potential for infinite recursion bugs.

It does mean you can't have an anonymous function which recurses, but that's a relatively rare use case. And actually, if you introduce some kind of special syntax which refers to "this lambda", you can even permit direct recursion of anonymous functions. Mutually recursive anonymous functions would be harder to support–maybe not impossible to support, but such an obscure use case it probably isn't worth thinking about.

Re:Lisp, Smalltalk, and the power of symmetry (2014) by kazinator on Monday May 30, 2022 @01:18PM (#31561215)
The referential integrity dreamcompiler refers to isn't just for cycles in the structure, but for substructure sharing. Substructure sharing still occurs in a DAG, unless you rule it out also (e.g. with some strict one reference ownership model or whatever). In the absence of cycles, it is still useful to have a notation which says "these two objects in the DAG are actually the same node". If you implement that, the detection of cycles is almost free, because the assumption that a duplicate node appearing in the print is not a descendant of the previous one doesn't simplify the printing or reading algorithms very much.

In order to print this #1=(... #1#) you have to determine that the given object occurs inside itself. In order to print (#1=(whatever) #1#), you have to determine that the given object occurs later in a sibling expression. Knowing that #1# will not occur inside (whatever) doesn't help all that much with anything; it doesn't help you decide whether or not the #1= prefix needs to be printed.

If anything if the restrictions were reversed, it would be simpler: if the only substructure sharing that were allowed was backward references, that helps. Because then we know that the (whatever) object can only occur again inside (whatever), and not after it. To detect nothing but cycles, without caring about other sharing, we just have to maintain an object path across the recursion so that for every visited object we can ask "is this object already in the path from root to here?" If so we have a backreference.

The sibling case is harder. You can have situations like:

  ((((#1=(foo)))) (((#1#))))
Deep inside the left item in the list there is an object which occurs again deep in the right item. We need to put the objects we are visiting into a hash, which is pretty expensive.

> Just by making sure your primitives for constructing values don't permit cycles to be constructed.

But lambda calculus teaches us that if all we have is functions being passed to functions and binding arguments we can build a circular reference. See the middle word of the domain in the browser address bar. So you have to further dumb things down from there if you want to absolutely eliminate cycles in the object graph.

(You can have it so that you don't care about function-induced cycles (only the garbage collector does) by not having any printed notation or other API that traverses the internals of a function. The application can still build a graph using functions, but is "on its own" as far as providing its own functions for operating on it, and serializing it and whatnot.)

Re:Lisp, Smalltalk, and the power of symmetry (2014) by kazinator on Sunday May 29, 2022 @09:39PM (#31554687)
Keyword parameters may be a late development, but they are based on property lists. Property lists are so ancient, that Lisp 1 and 1.5 used them for storing bindings. For instance, a symbol having a SUBR (machine language routine) or EXPR (interpreted routine) binding was stored in that symbol's property list.

Property lists and keyword arguments aren't tied to CLOS, though of course CLOS uses them for the flexibla constructor arguments passed to make-instance.

Re:Lisp, Smalltalk, and the power of symmetry (2014) by mpweiher on Tuesday May 31, 2022 @11:23AM (#31570596)
> and they don't have any kind of "class slot"

Yeah, that really sucks. Objective-S's object literals do. Otherwise they really wouldn't be object literals, would they?

It was really an accident, because Smalltalk's literal arrays use #(), and since I wanted to have dictionary literals use the basic same look, I chose #{}. And also I use {} for blocks and block structure, rather than []. Well, it turns out you can tuck a name between the # and the {}. Yay! Classes. And thus objects!

EDIT:

Just saw that edn uses a very similar mechanism. Neat!

Re:Lisp, Smalltalk, and the power of symmetry (2014) by srcreigh on Sunday May 29, 2022 @08:27PM (#31554281)
This is equivalent to an association list (aka alist) which is a technique which goes back to JAM.

    (define xs `(foo (bar ,1) (baz ,0)))
    (second (assoc 'bar (rest xs)))  ;; 1
    (define (prop xs k) (second (assoc k (rest xs))))
    (prop xs 'bar)  ;; 1
Re:Lisp, Smalltalk, and the power of symmetry (2014) by skissane on Sunday May 29, 2022 @08:56PM (#31554457)
Association lists have some obvious downsides – if they didn't, Common Lisp-style keywords would have never been invented.

One is that keywords are simpler syntax – `(foo :bar 1 :baz 0) is easier on the eyes than its association list equivalent.

Another is that their native implementation (linear search) can be a performance drain if you use them too heavily in an application (especially if you have too many keys). It is a data structure which optimises for insertion over retrieval, but most applications retrieve much more than they insert. The ability to "shadow" (have multiple bindings for the same key, with the first being used and the subsequent being ignored, but potentially becoming active again when the earlier binding is removed) is useful in some cases, but more often than not an unnecessary feature, more likely to induce bugs than be genuinely beneficial – "shadowing" by accident rather than intention.

A sufficiently smart runtime could optimise around some of these issues (this looks an association list, I'll use some optimised data structure to store it, but fall back on a more traditional cons-cell based storage if you try to do with it things the optimised data structure doesn't support.) But, having an actually separate data type enables equivalent performance optimisations, without anywhere near as much "smartness" in the runtime (which can be difficult to understand and maintain.) Plus, that kind of "smartness" can lead to difficult to understand performance regressions (why does adding this new code suddenly make completely unrelated parts of the application a lot slower?)

Re:Lisp, Smalltalk, and the power of symmetry (2014) by samatman on Monday May 30, 2022 @12:30AM (#31555497)
Lua's tables with metatables is a particularly eloquent solution to this, because as you say, it's always clear when a pair is being shadowed vs. replaced.
Re:Lisp, Smalltalk, and the power of symmetry (2014) by rileyphone on Monday May 30, 2022 @12:01AM (#31555352)
What you're describing with parameter ordering is Lisp's Mysterious Tuple Problem [0]. It's also part of why Lisp works so well for one or only a few developers, but breaks down for large teams. Javascript (and Self before it) has really started a structural revolution with the JSON object as a syntactic form. I'm currently working on a language very similar to your last paragraph, where all objects do have such a field indicating their class, so an object in the textual representation might look like:

  { Point x 2 y 3 }
Instead of n-ary tuple operation like Lisp, my design limits all functional application/message sends to a form of a receiver object, message, optional it object, and additional map of args in the preceding syntactic way (still not sure if they would have a class or be classless). Multiple dispatch would be limited to just the receiver and the it object.

You can see in Smalltalk where this syntax would be helpful in the awkward selector syntax it adopts.

  bool ifTrue: [block] ifFalse: [block]
Looks better as

  bool if {
    If
    $then block
    $else block
  }
Where here the $ in slot names means the corresponding value isn't evaluated, a reference to the late John Shutt's Kernel [1] that explored replacing the lambda calculus at the base of Lisp with a lazy vau calculus in order to restore the historically maligned fexpr [2]. Originally they were removed from the language because they made compiling difficult at a time when performance really mattered (1980), and replaced with special forms that weren't first class subjects of the language (and neither really were the syntactic macros that accompanied them). Going back to the original topic, Alan Kay listed Lisp as one of the primary inspirations for Smalltalk, and especially the fexpr, which he saw as underappreciated:

> There were not just EXPRs (which evaluated their arguments), but FEXPRs (which did not). My next question was, why on earth call it a functional language? Why not just base everything on FEXPRs and force evaluation on the receiving side when needed? I could never get a good answer, but the question was very helpful when it came time to invent Smalltalk, because this started a line of thought that said "take the hardest and most profound thing you need to do, make it great, and then build every easier thing out of it". That was the promise of LISP and the lure of lambda—needed was a better "hardest and most profound" thing. Objects should be it. [3]

I'm still in search of the "hardest and most profound thing" - here's to hoping it's real.

[0] https://www.codeproject.com/Articles/1186940/Lisps-Mysteriou...

[1] https://web.cs.wpi.edu/%7Ejshutt/kernel.html

[2] https://fexpr.blogspot.com/2011/04/fexpr.html

[3] http://worrydream.com/EarlyHistoryOfSmalltalk/

Re:Lisp, Smalltalk, and the power of symmetry (2014) by skissane on Monday May 30, 2022 @01:34AM (#31555755)
> { Point x 2 y 3 }

I like your syntax, but wouldn't it be better as something like

  { Point :x 2 :y 3 }
or maybe:

  { Point x: 2 y: 3 }
that little bit of extra punctuation (only needs to be a single character) is really helpful in making clear what is the field and what is the value. Syntactic minimalism is great, but sometimes it is taken a bit too far

> Where here the $ in slot names means the corresponding value isn't evaluated

If parameters have type declarations, why not make laziness part of the type instead of part of the name?

I suppose it makes it clear at the invocation site that the parameter is lazy-evaluated. I wonder how important that is though? In Smalltalk, it is very clear, due to [] vs (). In most Lisps, you can't tell from the invocation site whether some symbol is an ordinary defun, or a special form or macro, you have to look up its definition or documentation. In my experience, that isn't a big problem – if you don't remember it, sooner or later you end up having to look it up anyway, to understand what the parameters mean and what it actually does.

Re:Lisp, Smalltalk, and the power of symmetry (2014) by rileyphone on Monday May 30, 2022 @02:21AM (#31555970)
I agree my syntax is quite spartan, though that's because I don't intend to rely on it as the user interface, but would rather extend to structured editing more like Self.

On the laziness part I'm torn - quoting is what that amounts to and it never seemed extremely elegant either. The call site disambiguation is nice for knowing whether you can compile something.