What are you using Lisp for lately?

I'm not doing a lot of coding nowadays. But for the past five years or so I have had a consulting gig helping to maintain and extend a tool for VLSI chip design written in Common Lisp. Unfortunately, I can't say anything more about that at the moment.

Before that I wrote an e-commerce system that I used to run my company, Spark Innovations. It used Stripe for billing, Easypost for shipping, and Hunchentoot as the web server. I also wrote a spam filter that is still in production on my personal web server.

What does the Common Lisp consulting market look like?

If one puts themselves out there (build reputation on the Internet) as a CL programmer, are they likely to land consulting gigs?

Well, Clozure Associates was in that line of work and they are out of business. So that's a data point.

There are few job announces, but positions are found thanks to one's reputation and online presence. It definitely happens, that's a data point.

Thanks for writing this followup, Ron.

This paragraph really resonated with me:

> All this is a reflection of the so-called Lisp curse, the fundamental problem with Lisp -- its great strength is simultaneously its great weakness. It is super-simple to customize Lisp to suit your personal tastes, and so everyone does, and so you end up with a fragmented ecosystem of little sub-languages, not all of which (to put it mildly) are particularly well designed.

I have a career-long love and admiration for Lisp, but I've never used it for "real" things because of this precise reason. I wanted my code to be understandable and modifiable by others who are already familiar with the programming language I'm using. With Lisp, it's almost guaranteed that any sufficiently large program will be written in its own dialect no one but the author knows.

nah, disagree, how do you come to the conclusion it is "almost guaranteed" without having tried it for "real" things? (no mean for ad-hominem, I'd like to read details)

It's also a great language where you can adapt anything, any library, to your needs, bypass its shortcomings, etc.

What is the ecosystem of fragmented sub-languages? I don't know any in CL. (ergolib isn't a crazy sub-language either, it's just a library with helper functions and macros). We have solid libraries like Serapeum, supporting all existing implementations. Not sub-languages.

I agree in the weakness that it's too easy to fix a library and not contribute the fix back. Individuals and companies are guilty here. I wish I saw more companies contributing to the open-source libs they use. But other persons send great contributions. It's a small world, so we easily see the good and the ugly.

I believe the OP's ergolib provides an example. From https://github.com/rongarret/ergolib/blob/master/core/bindin..., the examples show code like:

   ;;; (bb
   ;;;  x 1
   ;;;  :db (y z) (foo)
   ;;;  :mv (a b c) (bar)
   ;;;  (do-something)
   ;;;  :with open-file f "foo"
   ;;;  (do-something-else))

And the code this replaces. This seems to me hard to understand without first digging into ergolib and grokking what bb means. It's just one simple example.

I forget now where it was, but there's a famous Lisp quote which goes something like (paraphrased from poor memory): in Lisp you don't just solve problem X, you first grow a new programming language designed for solving X-like problems and then simply solve X with that.

This has its charms, for sure. But it makes the code harder to understand for folks not familiar with your "language designed for solving X-like problems"

It's a little helper, so as in any language we have to check the doc and an example. Just like how to use a rest api, or a high level function that does a lot of stuff under the hood. Once you know it's a binding macro it's even easy to guess. I don't find it difficult, it is similar to many similar macros out there.

so yeah, I don't find the example extraordinary enough to speak about sub-languages. Plus, as another comment said very well, the most important is domain knowledge to understand the code. And the quote is right but we can also use Lisp like a dumb language and get shit done!

You seem to imply in TFA that you believe Clozure Common Lisp specifically is the best Lisp distribution. (Please correct me if I’m misinterpreting your stance here.)

What about CCL makes you describe it as a “technological marvel”? What advantages have you gotten from CCL that can’t be obtained from other Lisp distributions?

I’m asking as someone curious about Lisps and who always thought that the various Lisp distributions were roughly equivalent — so I’m sincerely curious, not doubting you.

> What about CCL makes you describe it as a “technological marvel”?

The fact that it started out its life as 1) a full CL implementation 2) with an IDE that is still usable today (you can run the original CCL on an emulator and it is till competitive) which 3) included an interface builder that was super easy to used, in fact easier than anything that exists today AFAICT, and 4) ran on a machine with 1MB of RAM and an 800k floppy disk.

The other thing that CCL had/has was a blazing fast compiler. It was/is fast enough that it was/is practical for me in many cases to recompile my code from scratch every time I run it.

When reading your section on Python and ending statements with "pass", and after not finding other references to it in your blog, I have to ask about Ruby. It seems like it would check many boxes (highly extensible, no significant whitespace, begin-end blocks, quite lispy...). If you've played with Ruby, what drawbacks have you found?

What are the major advantages you found in Python and in what cases do they exceed those in CL (aside of e.g. the type system you wrote about in 2008)?

I've never written any Ruby code but I did look into it once. What turned me off was its extreme approach to object orientedness. If I want to add two numbers I want to be able to write (+ x y) or x+y. I don't want to have to write x.add(y). That's weird syntax in service of an ideology that I don't subscribe to.

There were a couple of other weird things that turned me off but I don't offhand remember what they were.

This doesn't mean I think Ruby is a bad language, just that it had some in-your-face features that didn't resonate with me.

? Are you sure you're not confusing Ruby with something else?

x+y works just fine in Ruby.

Yeah, I was misremembering the details. But here's an example of the sort of thing that I don't like.

If I want to convert a number to a string, I do:

    x.to_s

which would lead me to think that to take the square root of a number I would do:

    x.sqrt

but I don't. It's:

    Math.sqrt(x)

So for every operation I have to remember whether it is invoked by x.op or op(x). In Lisp I don't have this cognitive load. It is always (op x).

That is not unique to Ruby. What is unique to Ruby is that the x.op style is applicable to literals. So you get weird things like 3.14159265.to_s which just looks bizarre to me.

Although the early versions weren't so, after the replacement of the base object model (see new style classes), in Python everything is an object as well, and many seem to miss that.

    print(type(1))
    print(dir(1))

produces,

    <class 'int'>
    ['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'as_integer_ratio', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']

That's true, but actually using a literal as a object is forbidden by the syntax.

    Python 3.8.9 (default, Mar 30 2022, 13:51:16) 
    [Clang 13.1.6 (clang-1316.0.21.2.3)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> 1.__abs__
      File "<stdin>", line 1
        1.__abs__
          ^
    SyntaxError: invalid syntax
    >>>

Ruby allows this:

    irb(main):001:0> 1.2.to_s
    => "1.2"

I think that's weird.

Only because those are special methods only allowed to be directly called by the runtime, doesn't change that the language model is actuall OOP based.

You could have used one of those that is allowed instead, example:

    >>> (3.2).is_integer()
    False

The parentheses are needed due to parsing precendence.

So here is your Ruby like example in Python.

Fair enough. I guess I should probably take another look at Ruby.

Like the other comments said, (1).__abs__

or

a = 1

a.__abs__

works

Thanks for answering my earlier question. Here's hoping you catch another one!

In a lot of my clojure scripts, I forget the shape of the data, and so to help future me, I've use tricks like pre/post constraints, defstructs, schemas, etc. In my experience, TypeScript has the best typing ergonomics (block-local, partial interfaces!). But since I prefer lisp language and interaction ergonomics, I'm always on the lookout for new developments, like https://github.com/coalton-lang/coalton, which has pushed me to start playing with sbcl.

Maybe CCL has already solved this, but I'm not versed in CL, so what methods do you find most effective for passing down knowledge of constraints and data shape to your future selves?

That's a really good question. No, CCL has not solved this. CCL is just a CL implementation with a really fast compiler and a nice IDE, but it doesn't really extend the language. Ergolib, which has nothing to do with CCL (it works on most CL implementations) extends the language, but it doesn't really deal with the "shape" of data. I haven't actually heard that term before, but I presume it's referring to data type constraints. If that assumption is correct, then the answer to your question is that this is a very hard problem. The solution that is currently in ergolib is not even half-baked. I had a vision of evolving ergolib into a fully-fledged new programming language which would be called Ergo and include a state-of-the-art type inferencer, but I have yet to get around to implementing it. It turns out that dealing with the vaguaries of external data sources is MUCH harder than dealing with inconsistencies within my own code. I hardly ever get type errors in my code, despite the fact that I have minimal support for typing, and even what little there is I hardly ever use. The closest I get is to be fairly anal about creating classes for everything and avoiding punning cons cells like the plague. Punning cons cells is a vert common practice in idiomatic Lisp code, but I think it's a Really Bad Idea (tm). So, for example, I really hate association lists. Ergolib has support for abstract associative maps, which allow you to use an alist as the underlying implementation if you really want to, but the default is a hash table, and I find I hardly ever need to change that.

The hard stuff for me is dealing with external interfaces, which almost always comes down to parsing. This is one reason I love Lisp -- once you get your data into the form of an S-expression, a lot of the heavy lifting has been done and everything just becomes much easier. So any time I have to deal with json, or xml, or csv, or whatever, the first thing I do is parse it to turn into a sexpr, and that almost always turns out to be the hard part of whatever I'm doing.

Did that answer your question?

Yea, for the most part.

The problems you describe with external data are supposed to be clojure's forte, but clojure also makes cheap footguns because maps fill most of the use cases for alists / hash tables, and implicit nil punning is everywhere. Once data comes through the wire and is turned into a map, if there isn't an enforceable structure in code, I have to run the program to check the data's shape. Hardly new in dynamic languages, and there are many solutions (e.g. spec, malli, schema, just to name 3 for clojure), but I haven't seen one matching the ergonomics that TypeScript 3+ offers today.

Thanks again! Always enjoy insight from experienced lispers.