Because if it hasn't "succeeded", for suitable definitions of "succeeded" in 50 years, as an old fart approaching 40 myself my assumption is that there is a good reason. I was much more willing to believe the "everybody else is just stupid and can't see the obvious brilliance" reason when I was younger, but as I've learned more, I've learned just how many factors go into a language being successful. Lisp may have all the features, but there are also good reasons why it has never really quite taken off.

That said, it is also true that using languages that bodge Lisp-like features on to the side after several decades of usage is generally inferior to languages that start with them. It's one of the reasons I'm still a big proponent of people making new languages even though we've got a lot of good ones. (I just want them to be new languages somehow; a lot of people just reskin an existing language with a slightly different syntax and then wonder why nobody uses it.)

But they've all been listed before, and I don't expect me listing them here will change anything, so I'll skip it here.

(I did a quick check on Google Trends; Clojure seems to not be growing. It's not shrinking, but it's not growing. That was Lisp's best chance for growth I've seen in a long time, and that window has probably now closed.)

Clojure has a different problem. It is based on the JVM infrastructure which was (imho) an unfortunate decision. Access to Java features is nice but dealing with Java stack traces is not fun. Also the usual startup time of clojure apps in the range of seconds is not acceptable (Tcl/Tk apps start in a fraction of a second). AFAIK clojure is also not suitable for mobile app development. The clojure developers should have used their own VM, or they should have provided a Clojure/Lisp compiler for native compilation. LuaJit has demonstrated how incredibly fast a suitable VM can be.

Clojure is quite wisely based on the JVM because the idea was not to create a Lisp replacement or a new Lisp, but rather a practical Lisp. Some of the historical problems with Lisp and many other languages like Smalltalk were related to specialized hardware, development environments, and/or ecosystems. Clojure did away with most of these concerns by attaching itself to one of the largest existing environments.

Using the JVM was a wise decision that has/had many advantages not limited to:

- Ability to leverage an already huge ecosystem of libraries, tools, servers, etc. that are well-tested

- Already battle-tested and working package system - this is severely underestimated by some popular languages

- Justifying its existence as a tool along side other JVM languages within an organization, not a full replacement

- Existing, highly optimized JIT

- Reasonably fast, nearly for free

Originally there were some plans to expand more to other environments, for example the CLR, but the JVM got the most attention and in the end this was practical.

As for some of the real drawbacks:

- Clojure alienates Lisps zealots and people who could never grasp Lisp. IMO, this is a stupid person/psycho filter so I don't see it as a drawback but it's worth noting.

- Startup time as you note. Startup time sucks, but it has gotten better and there are workarounds that are the same ones you could use for Java apps traditionally. The mobile dev issue you note is somewhat wrong and without a huge explanation, one option is to more or less enjoy a huge part of Clojure via ClojureScript, or in other words, targeting JavaScript and using things like React Native.

- Garbage. This is an issue of a lot of things on the JVM and Clojure is no exception. You can work around this somewhat with specific coding practices if you need to, but yeah, Clojure isn't going to be good if you can't fathom an eventual garbage collection cycle.

- No TCO, which is mainly a JVM issue if I remember right. This would be great, but it's a tradeoff and Clojure has negotiated this somewhat by providing what I feel is more readable code than when I worked in certain Lisps.

- Some legacy baggage in the standard lib, for example Clojure.zip. Recently they've been more brutal about what can and cannot go into standard libs. Every language though suffers from this a bit.

Regarding developing its own VM, I think you again miss the point about practicality. If Clojure did this, it would have been even more years before its release. Moreover, comparing to Lua is a bad example as it is a very different language (yes, I used Lua professionally). Lua achieves a lot by really keeping it simple, and while there is merit to that, Lua leaves a ton to be desired which I won't get off-topic about here.

So Clojure could work better in its own VM, but then you'd lose the JVM ecosystem along with many other things. I personally would rather have the ability from the beginning to reach from a huge amount of libraries rather than have nothing but what other people writing the language provide or via crude things like calling back into C. There are many talks about all of this, many from Rich Hickey himself. I think you really missed the point of Clojure and I am more of the mindset that I am glad it exists and not in an ever state of flux so that I can use it today, get things done, and not have it relegated to some research language I could never justify in a workplace. And no, I am not a Clojure zealot, I used about a dozen languages in any given year depending on my project and interests. There's a lot I prefer in Lisp over Clojure, but I see Clojure as taking some lessons from various Lisps rather than trying to be the one true Lisp.

My humble two cents to the Clojure team:

1) You should implement a cache mechanism ("save-image") for native code so that at least the annoying startup time of clojure apps is gone. I wonder why Java doesn't support native caches to this day.

2) The weird Java stack trace problem could be solved by providing an individual stack tracer which is close to the source code. I know that this is not possible for Java libraries but at least clojure stack traces should be presented in a more convenient manner.

[1]: https://github.com/pixie-lang/pixie

I wouldn't call Clojure a Lisp; it's a Lisp-like language with some interesting ideas, but not really Lisp at all.

As to why Lisp has failed to take hold where other languages have succeeded, I'll use a G.K. Chesterton quote: 'Christianity has not been tried and found wanting; it has been found difficult and not tried.' It's not that Lisp itself is difficult; it's that Lisp is different from lesser languages, and people have difficulty learning something different from what they're used to. Lisp has not been tried and found wanting; it has been found different and not tried.

So … try it! You may be surprised.

"In 1972, a British scientist sounded the alarm that sugar – and not fat – was the greatest danger to our health. But his findings were ridiculed and his reputation ruined. How did the world’s top nutrition scientists get it so wrong for so long?"

Without advancing an argument for Lisp, I'd just note that it's possible - in any discipline - for something to be decades (or more) ahead of its time.

Of course, that wouldn't mean that everything old is avant-garde just because it said it was. But, if people are still or suddenly looking at it, and looking at it hard, I would at least take that as a signal to give it some due consideration. It's apparently fighting age, and winning to some degree.

I can't stand the mentality of old = bad or not popular = bad. Most of the time, like anything in life, things don't "win" for being the "best" technically or for the most merit. There are usually many factors at play, and marketing, misinformation, stupid people, timing, and more have a huge role. While some of these factors may be good reasons, for programming, many of them are rubbish. We go down the bad paths more than the good ones in computer science it seems.

There are countless technologies that were way ahead of their time and for various reasons didn't end up market leaders today. Lisp is one, Smalltalk is another. Even Object Databases and various permutations of "NoSQL" databases existed for a long time. CSP is yet another that is being "rediscovered" via Go, Clojure, and some other languages. The list goes on for hours. Whether it was/is hardware or many other reasons, these things didn't "succeed." Despite that, it doesn't make a technology or idea useless if it's a good fit for the task, nor does it make it worth ignoring, not learning, or improving.

If there's one thing I've noticed from my considerable years in software dev, it is that most people are wrong most of the time about most things. Look at science - the field has historically been full of naysayers, people who cling to the past for their own agendas, saboteurs, fools, politically or religiously motivated morons, and so on. If we just always went with what the masses say or for that matter, even the so-called "experts," we wouldn't have any scientific advancement at all. Looking back in science, we can also see that many people had discovered or nearly discovered quite a lot. Although somehow we eventually unearthed some of these discoveries, the advancements they created never came in their lifetime or even century or eon.

There's a lot wrong with modern Lisp today, but most of the foundations are solid and it gets pretty tiring of people using Lisp like a pejorative, especially if it's because they lack knowledge about it or do not understand it, or perhaps worse, have never used it.

The smaller and more focused feature set is probably one of the reasons its popular. There's also ergonomic reasons, which have a big effect on whether people who aren't initially fully committed tune out or blow up early on in their encounter with the language.