I don't believe in a global linear scale of language power ("the blub theory"), but I do believe that some language may be better than others for a specific task given specific constraints.
E.g. if I have equivalent levels of experience in C++ and Python, I'm pretty sure I can write small webapp quicker in Python. OTOH if the languages are very similar, like Python and Ruby, the level of experience is much more important that the relative strengths and weaknesses of the languages.
Of course, it is seldom that you get that kind of fair comparison between two languages - usually everyone has a favorite language they know better than any other.
Another issue is that, if you are talking about small-to-medium sized applications then clearly there is a difference in languages. For example, it is pretty clear that writing a script is easier in perl than in C, or writing a medium sized expert system is easier in LISP than in Pascal.
However, if you consider large scale applications (100k+ LOC), then I don't believe there is any difference between writing in C, C++, Java, or Common LISP, as long as the programmer(s) have deep experience with the used language.
Just notice that the language is not going to solve the large-scale problem by itself. As long as such language has tools for creating abstractions, the code will have about the same complexity no matter what. If that complexity will be encapsulated in simple concepts (structs and functions) or higher level concepts (closures and continuation) depends on the taste of the developers and the language used.
The choice of abstraction do matter. If you use weak ones, your productivity is taking a serious hit: your program will be bigger, more complex, and have more errors (squared).
C++ abstractions, for instance, are incredibly weak. Take the function abstraction, which isn't even complete: you have no way to write anonymous functions the way you write literal integers[1]. Higher level concepts, as you call them, aren't more complicated than the "simple" ones. Often, they are just less familiar and more consistent.
[1]: Anonymous functions should actually be called "literal functions":
(fun x -> 7 x + 42) -- a literal function
357 -- a literal integer
2 + 3 -- expression which yields a integer
f . g -- expression which yields a function
Nothing "high order" about that. This is just acknowledging that functions are mathematical objects like any other.There are two kinds of design patterns: architectural patterns (e.g. MVC) and language patterns (e.g. Iterator). In the language I use for work, C#, we have to use a lot of both kinds of patterns. Often the "all code must be in a closure or a method" way the language works gets in the way (I can only imagine it's much worse in Java). In 100k LOC, I bet 25-30% of it (random "from the gut" guess) must be language patterns (e.g. visitor).
When you realize that nearly none (if any at all) of the language patterns are needed in Lisp you realize that in 100k LOC you only work with the problem. It seems to happen to me pretty often that I run into situation where there are two possible representations of something, both having problems and I realize that in Lisp I wouldn't have even noticed the situation at all because I could have used a more natural solution right from the start (CLOS' generic function approach makes all the difference in the world here).
Keep in mind I've used C++ and its descendants longer than I've used lisp.
I've worked on heavily used programs in several languages (including Lisp) and frankly in my experience the quality of the programmer is way more important than the framework they start from.
Can you provide a source for the 30% figure?
So features that are mostly implementation-dependent, like incremental development, static type checking, contracts, CPAN/gem/easy_install type code repositories, reference books and mailing list discussions, etc. are the things that give a language value. Exposing powerful key concepts(like common data structures, garbage collection, reflection, macro systems...) directly through the language is also a win but not something impossible to work around.
To paraphrase a oft-abused quote, "every sufficiently complex program contains a Lisp." But that doesn't mean the conclusion is "start with Lisp, since you'll end up with one anyway." It may be that Ruby (for example) has a really cool library you want to use for a core part of your app, so you start writing the code in Ruby, and things are good and you make progress....and once you come across a problem requiring Lisp-type power, just by knowing how the solution would work in Lisp, you can usually devise a worse-is-better 80% solution that is right for your specific application. The final result may not be beautiful or pure, but it lets you do things incrementally without the up-front misgivings of "it would have saved so much time to start in Ruby..."
tl;dr: Learn languages to learn concepts, but build applications in an "environmentally-friendly" way. Perceived "potential power" is not a good reason to sweat and strain to build apps in your favorite language when you could go over to the current "industry standard" and save yourself months of effort.
On the other hand, nobody has an omniscient view of all languages and environments, hence people tend to stick with what they know....
Not quite - there are a bunch of languages that are "up there" in terms of expressivity: Lisp, Smalltalk, Haskell, ML, etc. There's no data to say "learning Lisp will give you an enormous productivity boost". There is only data to say "people with 30 years experience in Lisp/Smalltalk/whatever tend to be very good programmers". And, well, duh.
Also, if he had to do it in C, say, could he really not? Even if he needed to invoke Greenspun's Law in the process...
The question is whether that always happens for a complex system. Because if it does, then every sufficiently advanced programmer is a lisp programmer.
I recently (~5 months ago) started using Clojure heavily and now find myself much more productive using it than using any of the languages that I have considerably more experience with (I used C++ almost exclusively for 4 years for hobby (mostly game related) projects and toy virtual machines/interpreters; Python for ~4 years for prototyping, web development and GUI development; Java for 2.5 years for server (non-web) development - total programming time including hobby, uni and professional = ~10 years).
As an example, I'm very much interested in compilers, virtual machines, programming language design and such, so have been plying around with these in various languages over the years. I wrote my first toy language in VB; wrote a few interpreters, virtual machines and simple parsers in C++ and some parsers, interpreters and assembly code generators in Python; for uni, I implemented a parser and code generator (instruction selection, using maximal munch) in JavaCC - recently I wrote an assembly code generator in Clojure. It took me a weekend and it surpassed the power of anything[1] I'd written in VB, C++, Python or Java. Its flexible and can be easily be extended, its pretty smart and can do some basic optimisation (caching data in registers, function inlining...). I tried writing something similar in Python before, but gave up.
My point is that I have much much deeper knowledge of C++, Python and Java, yet I was able to build something MORE complex and extensible in Clojure, in less time, even though I'd only been using it for a few months.
Yes, my previous experience played some part in this, but I attribute most of it to the flexibility and power Lisp provides me through easy to use and powerful abstractions, flexible and convenient syntax and interactive development.
[1] Of the same scale - any interpreter, code generator, compiler etc. I've written much more powerful systems, of course, but they are huge in comparison and took a lot lot longer than a weekend to write.
So, he probably could have written it in C++ (or SNOBOL ;-) ), but he chose to do it in Lisp.
It is amazing how much Python (and other high level languages such as lisp, Ocaml) allow you to focus on the problem you are solving not the irrelevant details of the solution.
... and we could not have done this without A, B, C ... by Edi Weitz.
Lisp needs more people like him who actually release software instead of blogging about its virtues.
People have asked Jack to explain why he used Lisp for an embedded application. You can read the result in the linked article.
The guy who has posted the story also has written a bunch of Lisp based internet applications. If you look among the hardcore Common Lisp users, quite a lot are writing software. The once that are blogging superficial stuff not so much. Read the stuff on planet.lisp.org - that's usually quite useful.
When you buy a piece of no one writes on the packaging "we did this in C++!" or "We did this in Java!" Why would you expect a big "Common Lisp!" sticker on the packaging? The deliverable is machine code, not source code. The customer doesn't care what the source is written in.
Ever read blog posts about those?
I started programming in LISP way back in 1971 on a Univac 1108 mainframe and also implemented a 68000-based Lisp system (~50K lines of real-time assembly) for mobile-robotics use in 1983 - and so know my way around the language.
All Lisp (or Smalltalk, or...) success stories I've read hinge on someone with an enormous amount of experience with the language. I'd argue that someone with that much experience could get the job done in (almost) any language. I'm surprised that someone with that much experience would put it down to language choice rather than deep knowledge of the problem domain.