> Answer: files, strings, and hash tables. All of those languages make mangling text pathetically easy. Perl is obviously the poster-child for one-liners, but all of those make that task pretty darn easy.

Fair point about strings, not sure about files and hash tables. But you've mentioned a bunch of scripting languages, and CL was never competing with these. Why it lost to Java and later to Python and Ruby in webdev is a different question. But string handling - you need to notice that as an industry, we shot ourselves seriously in the foot with Unix and proliferation of unstructured text. The unstructured text-oriented philosophy is frankly insane, and lisps are older than it - so for better or worse, they didn't follow.

People now are slowly rediscovering why adding structure to your text is better than having each program implement its own incompatible, bug-ridden regex-based parser. And surprise surprise, structured data is something lisps excel at since before most of us were born.

> But you've mentioned a bunch of scripting languages, and CL was never competing with these.

Except that it WAS. Programmers have a limited amount of mindspace. So, the question as a programmer in the late 1980's/early 1990's is how I'm going to allocate that space:

1) Systems language--C is pretty much your only option at that point unless you are in assembly (God help you). Anything bashing hardware (graphics, audio, networking, etc.) goes straight for C.

2) Many programmers are also system administrators in this time frame. So, I'm going to use a language that mangles text well--Perl wins here. People transition to other languages only once they get fed up with Perl.

3) Operating system specific languages. Unix pushes you toward C. Windows was C or Visual Basic (that ecosystem was huge at one point). Mac was C or Pascal.

So, in this time frame: Lisp is painful for system programming on small systems (fails 1). Lisp manipulates strings badly (loses on 2). Lisp is not tied to an architecture that would drive adoption (loses on 3).

So, as a programmer, Lisp is, at best, at slot 3 or 4 in terms of programming languages for daily use. Is it any wonder Lisp never got real traction?

You also have to consider how knowledge got disseminated in this time frame. The web doesn't exist. Email is actually scarce unless you are at a university or a big technology company. Netnews is something that soaks up enormous amounts of disk space so sysadmins hate carrying it in useful ways.

It is hard to appreciate, in this era of instantaneous, ubiquitous information, just exactly how much the pink camel Programming Perl book with its cookbook section singlehandedly drove adoption of Perl. I had 3 copies of that book that I used until they disintegrated. Can you think of ANY other programming language book for which that is the case? (Actually, I can, but I'm the odd duck. My workbook copy of The Little Lisper got the same level of abuse to disintegration).

> The unstructured text-oriented philosophy is frankly insane

It is NOT insane when memory and disks are small--which defines most of the time when Lisp was popular. Structured text requires lots of extra overhead that simply wasn't going to fly when RAM, disk, and bandwidth were very restrictive.

Very true. Now you mention it, I wonder why, given the awesome power of extensibility, this hasn't been improved? Shouldn't it be trivial to write a DSL that makes string handling almost, if not as good as, Perl et al?

It is, which is probably why nobody bothered to put it in a library before. But you have CL-PPCRE, which is arguably the most performant and one of the most solid implementation of Perl-compatibile regular expressions out there.

The only string related task I've found Common Lisp excelling at is it's print formatting syntax. That stuff is really nice.