It's unlikely that an open source language is both "secret", and has oodles of libraries - things just don't tend to work that way.
Here's a simple anecdote: Erlang doesn't seem to have a good image manipulation library/interface to Graphics/ImageMagick. That's something most popular languages have at least one of.
That said, when you have a task that's a good fit for Erlang's sweet spot, Erlang is very good. The runtime is a very fine bit of software craftsmanship.
That is true, but, at the same time, irrelevant. For example, at my company we do e-mail archiving. Does Erlang have any libraries to facilitate that? No. Not even close. But Apache Lucene does. So, what we do is use the Erlang OTP Java interface [1] to allow Erlang to handle the message passing and parallelization, while Lucene handles the actual indexing and search.
The way I see it, Erlang makes it very easy to add a message passing layer on top of whatever single-threaded process you currently have. This allows you to scale your applications without having to muck about with their internals too much - you launch multiple instances of your existing single threaded code and use Erlang to handle all the messiness of message passing.
I think that is what makes Erlang such a secret weapon. The fact that you can very easily add a message passing layer to anything with Erlang greatly simplifies the job of parallelizing existing programs.
[1] http://www.erlang.org/doc/apps/jinterface/java/com/ericsson/...
Ordinarily this would be a problem, but ImageMagick (since you mention it), in my experience, is something you do not want to link directly to. Despite being a quite old and (theoretically) mature project, it's quite brittle, leaky and buggy, prone to segfaulting and weird freezes. These days I just fork and exec the `identify` and `convert` libraries to do image conversion (and simple manipulations), and for graphics drawing I use other, less buggy libraries such as Cairo.
It really depends on the history and the typical domain that language is used in. Yes, Erlang doesn't have good image manipulation library, but it has built in support for a multi-node distributed system setup. Applications with automatic failover running on 2 nodes. Also a distributed database and so on. These are not trivial things to just whip up on other languages, but Erlang has them in the standard library.
It has also been used for telecom applications for a long time before it became open source, and thus it accumulated a decent size library over the years, which it might not have had it been a "secret" language started from scratch by a PhD student 5 years ago...
- Actor Model
- Pattern Matching
- Hot code replace
Yet HotSpot is more performant VM then erlang's VM is. Now, if those features could be implemented on top of the JVM it would (to me at least) render erlang to be irrelevant.
So let's examine what is currently known.
- Actor Model: An example of a massive implementing effort of Actor model would be scala's Akka framework http://akka.io/
- Hot code replace: I don't know the answer to whether HotSpot lame code swapping implementation can be fixed to act like erlang does it. http://scala-programming-language.1934581.n4.nabble.com/how-...
- Pattern Matching: As for pattern matching, to my understanding this can be implemented on top of the JVM, if one wishes to. http://news.ycombinator.com/item?id=2784086
Not to mention Erlang's live console and logging capabilities which are indispensable when it comes to diagnosing crashed services or inspecting live systems and gathering stats. The fact that Erlang's data structures are immutable allows OTP to give you a very detailed report containing the exact state before the crash and the message that led to it, while in Java and most other languages your best bet is just an exception trace - and then you would have to guess what exactly went wrong because you don't have the system state anymore and can't recreate it easily for that matter. You can imagine that this makes debugging an Erlang application a joy - at least as joyous as it can get.
One other important aspect of OTP are supervisor hierarchies - Erlang is built around the idea that if something can fail, it will - so systems should be designed in such a manner that failure in one component doesn't tear down the whole system. Services are carefully subdivided into autonomous submodules, and if a module crashes OTP will attempt to restart it and continue its operation if possible. Imagine a png conversion submodule crashing because of a faulty png header - in a Java system this would most probably throw an IndexOutOfBoundsException and down comes your whole backend. Erlang/OTP would just note that the converter has crashed and then restart it. It can do that because the converter doesn't affect or depend on some interconnected, global, "complected" state and thus can be easily started and stopped at will.
Of course, Erlang's solution to concurrency is no silver bullet - you can still create deadlocks if you aren't careful and you can still create global state by abusing named processes, but it makes a lot of things a lot, lot easier.
edit: one more thing - Erlang's hot code swap support is really one of the most beautiful things I've seen when it comes language features. Now, swapping code by itself isn't such a big problem - you can do this in C as well by just reloading a DLL or in Java by un/reloading a specific class - but the tricky part is about swapping state, and this is where most imperative languages fail hard.
Said otherwise, when you update your service, you not only want to update your code but also your data structures. For example, if you have a class/struct like this:
struct person {
string name;
int age;
};
and would like to add an additional field `ageString` to it, for whatever purposes, how would you go about doing this in a more conventional language? You would have to track all usages of `person` somehow, and then replace the in-memory representation with the new version while making sure that no thread is reading or writing data, then swap the code while being careful that no new code is reading old data or vice versa. In fact, in most systems I've worked on such a thing would be almost impossible.Erlang's processes and immutable data structures allow you to do just that - and as painless as possible. You provide a function that converts old state to new state (in this case containing the field `ageString`), the new code which operates upon the updated data structures and let Erlang do the rest. All messages that come in while the swap is taking place will be queued up and delivered later.
Moreover, if the update introduces a bug, you can as easily downgrade as you did upgrade. And all this while your service is up and running!
I love Erlang, but it's not all roses either.
And fault tolerance is exactly what JVM doesn't have. It shares a common garbage collector and there are other things.
That is the main problem with those who claim "oh here is a faster Erlang library in language X", yes, one can copy some of the features into language X but unless full process isolation is copied, it is not really an Erlang replacement library/lanuage.
Systems are not divided into cruise missile control systems and everything else. There is a continuum in between. A phone system where the calls need to routed in the 100 milliseconds or less 99.999% of the time is acceptable for example, while one where no such guarantees can be provided at all is not acceptable. It is not a realtime systems but it is pretty close, so it would be called soft realtime.
It suspends your process, then logs into newegg.com and buys more memory sticks for your sever. You know it is just a regular VM and doesn't have magic in it. If you use up all the memory on your server you are screwed probably.
You could set up a monitor to see which erlang process is eating memory and what rate and then decide what to do, kill the process, restart the whole node, or report someplace.
I have been playing around with it and it simplifies a lot of the things that bother me about erlang. It breaks some functional conventions for code readability such as rewriting the same variable
As a matter of fact, Clojure runs atop the JVM and has a concurrency model that parallelizes even better than Erlang's, although Erlang's model of concurrency is presumably more fault tolerant.
"While we make heavy use of the core of Clojure, we don't use its concurrency primitives (atoms, refs, STM, etc.) because a function like pmap doesn't have enough fine grained control for our needs."
When the goal of concurrency is parallelization, I'd want the most abstract and powerful tool for the job, which for me means Clojure over Erlang. When the goal of concurrency is reliability, I imagine I'd want separate lightweight processes for fault tolerance, which is where Erlang seems to excel.
Erlang is never going to be the really "hot" or "cool" language because it has an unusual syntax and that syntax is enough to keep it from becoming fashionable.
But this is a feature, actually. Because the really good engineers, they will invest the week or two (really!) to learn the language and once they do that they come to love it, and as a result, erlang has some of the best engineers working with it.
What Erlang does, you simply can't do with a library or bolt on solution, or anything involving ruby, python or the JVM. And what erlang does-- concurrency done right-- is so valuable these days that anyone solving real problems has run into it.
Erlang as been quietly winning in production and building out just about every library you could want. And because the language is so elegantly designed, for many projects its really accessible- you can dive into the code and comprehend it. (I get lost in the riak sources, because it does so much it overflows my stack, but everything else is easy.)
Erlang is not only a secret weapon, but it is a productivity multiplier.