> Sometimes with lisp things get "spooky" in a way that just doesn't happen in practice with Ruby
Over here the long run practical experience with Ruby on a couple or three significantly involved codebases and assorted dependencies taught me over the years that such solutions are elegant for a month then end up being a hell to maintain, debug, and trace through. We definitely encountered my lot of "spooky" things in Ruby-land. For maintainable code on the long run we now vastly favour boring, non-dynamic, explicit, down-to-earth code. Dynamic solutions, DSLs or dependencies making use of those now therefore have to sustain solid justifications and heavy scrutiny before being greenlighted.
Sounds like your experience was with Emacs Lisp, a very clunky Lisp.
>Though I completely agree that inheritance gets clunky and I strongly favour composition.
Check out CLOS (Common Lisp Object System), which is based on multimethods/multiple dispatch. It is very different to "traditional" OOP (most other languages except Julia and Dylan)
Emacs lisp is actually a very practical, easy to use and fun lisp to do a lot of great work in.
Richard Matthew Stallman notes:
It was Bernie Greenberg, who discovered that it was (2). He wrote a version of Emacs in Multics MacLisp, and he wrote his commands in MacLisp in a straightforward fashion. The editor itself was written entirely in Lisp. Multics Emacs proved to be a great success — programming new editing commands was so convenient that even the secretaries in his office started learning how to use it. They used a manual someone had written which showed how to extend Emacs, but didn't say it was a programming. So the secretaries, who believed they couldn't do programming, weren't scared off. They read the manual, discovered they could do useful things and they learned to program.
From: https://www.gnu.org/gnu/rms-lisp.en.html
Of course if secretaries and regular office people with no prior exposure to programming or STEM education can program Emacs using Emacs Lisp and do useful productive stuff. Programmers most certainly can.
I was just trying to point out the fact that its not hard to code in Lisp. Especially given the fact that non-programmers were doing it long back, in an era where there was nothing like Internet available to ask for help.
I think you could make the argument that Emacs Lisp is as close. Emacs Lisp has some major similarities with Maclisp that CL doesn't share (like in variable scoping and obarrays and such). Elisp and CL are both very close relatives of Maclisp; Elisp was designed as a mini-Maclisp for a text editor, whereas Common Lisp was designed as Maclisp's successor.
The lisp that didn't win is not an acceptable lisp.
Effectively programming languages get less powerful the longer the program becomes. Breaking up programs into more powerful little pieces is never a clean separation and you need to make real trade offs between more separation and more power.
I have found that to be true in dynamically or duck typed languages.
I have found it to be not true in strongly typed languages with good tooling. Example: refactoring large java program in, say, Eclipse.
> Breaking up programs into more powerful little pieces is never a clean separation
I would tend to disagree, to some extent.
In Lisp(s), there are lots of small, sometimes obviously correct functions that do powerful operations on abstract data structures. Part of the reason for this is that unlike OOP (and I write Java daily) which has lots of data structures, Lisp has very few. In OOP you model your domain with new classes. In Lisp you model your domain with combinations of a few basic structures. (In Clojure...) List, Set, Array, Map.
Maybe you have layers of separation. With one layer being between primitives that manipulate your domain objects, and another layer being the logic that decides how and when to do the manipulations. I want to solve a puzzle. The puzzle board has various states, pieces, and operations that transform a board into a new board state. Then the logic layer is an algorithm or algorithms written using the layer of primitive manipulations. That logic layer may, in fact, use canned algorithms, because the data structures are common primitives. Say, sort, search, A* search, depth first, breadth first, reduce, map fn over a sequence, etc.
You can end up with a large program. But it can be a lot easier to reason about than, say, a large Java program. It seems to take more deliberate effort to keep a large java program easy to reason about.
Layers are exactly the kind of power trade-off I am talking about. If code X can't(1) impact code Y, you have less freedom in code X because it can't do some things, but can more easily reason about code Y.
(1) either as an actual limit or a self imposed one.
Granted, in theory you may be able to find perfect points of separation such that you don't lose power by doing so.
Can you substantiate this claim? One of the reasons Lisp was used in the past was because it was particularly suited to complex, big projects... Like autopilot for spaceship, wind tunnel simulation, CAD/CAM systems, etc.
Lisp (as 'Common Lisp', the traditional dialect) has this concept of "packages" and "systems".
You can divide your code in "packages", each package contains separate namespaces for classes, function names, variable names, symbol names, keyword names and others. Thus each package doesn't collide with another package and names don't collide at all.
Many packages are combined into a "system" (by defining, for example, the required order for compilation, etc).
Thus you cleanly separate your code in packages and systems. The 100,000 line program becomes a combination of many shorter programs.
Of course this is nothing new, this is just standard practice for keeping big programs manageable.
>Breaking up programs into more powerful little pieces is never a clean separation
Care to explain? Well, i'll continue with my post. And we'll assume that what you say is true, so i'll list other advantages to keep complex systems at check.
Now, here is a big plus for complex systems: In such a Lisp, the system is hot-patchable. You can redefine functions while your program is running. Without stopping the program. We're talking about doing this even on a production environment, if necessary.
You can also redefine classes while your program is running. Without stopping the program.
Another big plus is the condition-restart system, which very few languages implement. Lisp has not only the notion of "catching exceptions", but also after catching the exception, provide alternate ways of recovering of this exception, including being able to resume operation at the point where the problem was found.
This is another big plus for building reliable systems -- and complex systems benefit from being composed of reliable parts.
I could go on with other big pluses as well. For example, multi-paradigm. Some parts are more readable (and thus easier to understand, maintenable) if written in the functional style. Others map better to the OOP style. Others would map better to the imperative/procedural style. Other would better be expressed as logic programming (think Prolog.)
Lisp allows you to use all these paradigms. So the code stays clear. I say this is very good for complex systems.
Some of this comes down to what you consider a 'large' program. Suppose you want to build an MMO in lisp, now you need to consider things running on both a client, and server, as well as rendering, path-finding etc. Choosing reasonable primitives becomes difficult as you get pulled in several directions. Should I be running the same code on client an server or should I have multiple different models communicate etc etc.
My point is you start dealing with trade offs and no approach is universally better. Even with a clean API between different pieces of code you still get conceptual dependency's.
Building anything complex requires, apart from tools, getting organized in a lot of other things as well.
The is an old saying they enterprise software ends up mapping the organization that built it. If you have N teams you tend to break a large project into at least N parts.
Sharing code between teams works, but it's harder to share complex lisp macros vs functions. Which is really closer to my point, the larger the project the more tools get set aside too being to ~complex/risky to safely use.
And Common Lisp has arguably the most flexible and powerful OOP system around, CLOS.
Why don't write OOP in Lisp? It is done successfully by lispers around the world.
Because most people understand OO as some kind of namespace scheme. Basically to bunch together functionality under a name(class).
When people talk of OO, most of them, are actually talking about modularizing code.
It's a lot more than simply being a namespace. It's also that idea that an object has state and behavior, with appropriate semantics for dealing with both (depending on the language).
Objects (with classes or prototypes) act as a new kind of data type bundled with functionality for handling that data, which is more than just putting a bunch of functions and variables into a namespace. So you can treat your class/prototype as a built-in complex data type. You're extending the type system in an encapsulated manner.
It's not called Namespace Oriented Programming
Also, in any statically typed language, you can fall back on changing the function and using compile errors to tell you all the calls that need to be updated.
rename a function
Refactoring in software development is not the same as refactoring in IDEs. Refactoring in software development is changing existing code. Specially the architectural parts.
Well actually no. Other languages make it easier for beginners to get started, but they just keep you there. Because its harder to do that the advanced stuff in those languages.
>>Almost every problem elegantly solved in lisp has a parallel elegant solution in Ruby (using blocks, dynamic method definition, or otherwise) and I generally find the resulting code both more maintainable as well as more readable.
Its other way around. We solve trivial library stitching problems in other language, and the we try the same in Lisp and don't see the point. But then that is also a kind of mental handicap. You can do things beyond a point because tools are making it harder for you to keep going forward. And then you use Lisp with the same philosophy and don't see the point.
Lisp didn't win because it used to cost money to get a good implementation, and there was no company throwing tons of marketing at it.
Now, I specifically don't think it is a case of worse ones winning. The reality is they are all impressive. Many newer languages got to make tradeoffs that were not viable years ago. They are still nicely done and I'm glad they are here.
This was one of the reasons.
In the late 70s, cheap computers (IMSAI, Altair) were almost unable to run Lisp for useful purposes. Too little memory.
Minicomputers (DEC PDP /etc) were able to run full Lisp implementations but it worked slower than using other languages.
Lisp Machines (MIT CONS, etc) were able to run Lisp fast but those were dedicated, specialized hardware. And expensive.
Enter the late 80s; Lisp ran great on personal computers but implementations AFAIK were mostly commercial, so reserved to big budgets. I'd say Common Lisp did have success on the industry, used for many things (3D, CAD/CAM, simulation, etc).
Meanwhile the rest of the world was on Pascal, C, and C++.
Nowadays things are different, there are many Common Lisp implementations that are free and are good (SBCL and CCL, for example, are lightning fast).
But additionally, it is difficult to understand what advantages would Lisp bring. For this, the developer would have to grok (completely understand) the enormous value of metaprogramming, and to understand as well how flexible is CLOS.
And I agree that things are different nowadays. My main point is that this seems pretty compelling evidence to me. But, what it isn't, is a testable hypothesis. (Is it?)
You'll see many people in such languages say things as "please don't use macros", "metaprogramming makes things difficult", "metaprogramming is hard", "macros make the code hard to read" and so on.
Metaprogramming in all other languages is usually difficult to implement OR is limited to text substitution (example: C), which makes them a mess!
Languages with true macros mean the code gets transformed to an AST first; your macro needs to transform this AST. Thus you need to learn a whole host of functions/methods/classes and to learn the structure of this AST.
While in Lisp, macros are written using plain regular Lisp. There is almost nothing new to learn, really. This is because the language is formed of s-expressions, which are really easy to manipulate. So doing metaprogramming is almost no more difficult than writing a regular function. In Lisp, metaprogramming is not an advanced topic; quite the opposite, is a beginners' topic.
If such "other languages" wanted to make metaprogramming easy, then the language would have to be written in s-expressions.
Which means the language would probably be labeled as a "Lisp dialect", or as a "Lisp-like language."
>other languages are adding advanced features all the time
Yes, but in Lisp you can add them yourself, you don't need to wait for Google or Oracle or your Benevolent-dictator-for-life to approve the features you need, and don't need to wait for the next version of the compiler.
So sorry, most languages don't have the features lisp has. Not even remotely close.
I think the concepts are great, and it's a great language for certain task, but my right pinky finger hurts just looking at it.
New programmers and developers look at that and compare it to something like Go, Python, or JavaScript, and all those look easier to write (although deceptively complex in places) and are 1000x easier to read.
A prettier lisp would do the world good, but things like sweet expressions are confusing for absolutely new devs to implement and veterans feel like they don't need them.
Short version: Lisp wallows in obscurity thanks to it's "look"
Are you sure?
Reading a file in Go
func read(f) {
var text string
var scanner *bufio.Scanner
var err error
file, err := os.Open(f)
if err != nil {
log.Fatal(err)
}
scanner = bufio.NewScanner(file)
for scanner.Scan() {
text += scanner.Text()
}
err = scanner.Err()
if err != nil {
log.Fatal(err)
}
fmt.Println(text)
}
in Common Lisp (defun read (filename)
(with-open-file (input filename)
(loop for line = (read-line input nil)
while line do (print line))))
Now tell me which one is easier to read. And the lisp example you can understand what it does without even knowing any kind of Lisp language; it is almost english.Perhaps a lisp with python-style indentation for blocks would be more easily read?
def read (filename)
with-open-file (input filename)
for line in (read-line input nil)
while line do
print lineGranted, the bike shedding that goes on in our industry is hilarious. I think many people blame semicolons as one of the major hurdles to learning c nowadays. The things we fixate on...
Secondly, parentheses are a non-issue in lisp. Its almost like reading english text with spaces. After a while(quite quickly in-fact) they are a total non issue.
>>New programmers and developers look at that and compare it to something like Go, Python, or JavaScript, and all those look easier to write
Arguably the biggest problem today is programmers who are stuck at early intermediate level all life.
> if the next line is indented, anything from the beginning of the current line starting at the indent till a dedent matching the current line indent is wrapped in one set of parentheses.
Seriously, what about this?
defmacro deftag [name]
list 'defn name ['& 'inner]
list 'str "<" (str name) ">"
'(apply str inner)
"</" (str name) ">"
deftag html
deftag body
deftag h1
deftag p
Now what about the DSL that now downright looks like slim-lang?: html
body
h1 "Hello World"
p "How's it going?"
Or look at that: defn qsort [coll]
if (empty? coll)
coll
let [pivot (first coll)
remainder (rest coll)]
concat
(qsort (filter (partial > pivot) remainder))
[pivot]
(qsort (filter (partial <= pivot) remainder))
(even the qsort ones could be removed, but they balance [pivot] somehow)I'm perfectly happy with writing (+ 1 2 3) or whatever, the only parentheses that bother me are those that are perfectly delineated by indentation already, just as newlines perform the same duty as semicolons. They're just duplicated noise††.
† Other languages perform similar semicolon insertion, with varying degrees of success and ambiguities (JS==terrible, Go==terrific).
†† I know, you're supposed to get used to it and "unsee" them. I know, "paredit!". But seriously if you have to write a tool solely to produce noise, and you have to get accustomed to ignore said noise, well, somehow, something's off.
Modern incarnations of the same like F# are attractive too.
But Lisp is what it is, List processing.
For example, see https://gist.github.com/coding4food/1248505
Part of this is because lisp tends to be more concise, so there are fewer expressions required. There are relatively few places where lisp adds parentheses, rather than just moving them - mostly just basic operator expressions: (+ a b) vs a + b. Adding parens (a + b) doesn't make it any harder to read.
For function calls, Algol-descended languages already use parentheses and prefix notation, they just put the parens around the arguments instead of the whole expression:
(function argument list)
vs function(argument, list)I've used Clojure professionally now for a year, and I'm not finding the problem you mention to exist. Maybe its exclusive to emacs lisp?
I would say it's more fundamental, simply a large majority find it difficult to interpret and reason about trees, rather than sequential operations.
There is a reason lisp didn't win. We put up with it (I use emacs, so I have to write it at times) but in general it's harder to reason about transformable lists of code than it is about plain old objects and their methods. Almost every problem elegantly solved in lisp has a parallel elegant solution in Ruby (using blocks, dynamic method definition, or otherwise) and I generally find the resulting code both more maintainable as well as more readable. Sometimes with lisp things get "spooky" in a way that just doesn't happen in practice with Ruby, even if it theoretically possible to say redefine String#inspect. Though I completely agree that inheritance gets clunky and I strongly favour composition.