I've heard talk that things start to get a little hairy once you scale to around 100 nodes, due to the sheer amount of messages getting passed around (there are ways around this, like creating hidden nodes that communicate between clusters). Performance across multiple VPS systems is really going to vary dramatically between different applications and VPS setups. An app that has to make lots of requests to other nodes, which are located across the globe, is going to suffer a lot more than one that has relatively little internode communication between VPSs in the same server rack, but that's just common sense. I'd be really interested to know what companies like Basho (creators of Riak) and Facebook have experienced with large setups, as horizontal scaling is one of the language's selling points.
More so than scaling across multiple cores and machines, I find that one of its strongest points is it's scheduler. If one request to a server is particularly processor intensive (or hangs), it's not going to affect other requests, which is an awesome feature to have even with a single core.
Erlang's definitely not for everything, but once yo get the hang of it, it's really easy to prototype in, scales well, and is generally lightweight enough to run on a cheap cloud instance (or embedded on a Raspberry Pi).
Here are some benefits to single core erlang, off the top of my head.
* Erlang's VM is phenomenal at handling concurrent I/O even on a single core. Free epoll/kqueue backed async i/o and iovec ops.
* OTP, applications, supervision trees, state encapsulation, fault isolation, cheap GC.
* Built-in distribution, node/process monitoring, code hot-loading, mature tools and introspective capabilities on runtime state/performance.
* Some degree of "free" scale when a company is ready to deploy that app on a machine that has more cores.
I'm not sure what you mean by requires a lot of RAM up front -- compared to C/C++, yes.. but, relative to ruby, python, or the JVM? That's not been my experience.
I'm not arguing against Erlang. It's a great language and folks could certainly choose to run it on their own multi-core system or scale across multiple virtual servers. But startups often pick the technology they can get running with minimal investment, then scale from there: a VPS or two with a single CPU and constrained RAM. So although Erlang might be a great long term investment, it requires a lot of RAM up front and hyped features like multi-core support don't matter as much to young startups choosing technology at the early stages.
It would be great to see more discussion about how well Erlang scales across multiple VPS systems -- and not just theoretically, but how many systems you can practically scale across, how it impacts performance, RAM usage, etc. Plus how this would compare to just using an independent messaging queue/bus to accomplish the same thing in another language.