> But something like "Pid ! Msg" where Pid is in the same OS process, different OS process or even different process on a different host half way across the world, will become a new way to think about programming.

That's why I started learning more about ZeroMQ http://www.zeromq.org and structured my latest work project as multiple processes instead of multiple threads. Each process is single-threaded, and (at least so far) relatively easy to reason about.

Good point. I forgot about 0MQ. Yeah not everyone will learn a new language but still want to build a distributed system. 0MQ is great for that.

With Erlang, which was perhaps confusing from my post, the Pid identifies an "Erlang process". It is a very light weight unit of execution. One can have tens of thousands of them in a single OS process (which they call a node).

However, if you don't need tens of thousands of processes, there is not reason why one couldn't structure their app using real OS processes and using 0QM.

For anybody interested in the topic, I would also recommend to read Joe Armstrong's PhD thesis: http://www.sics.se/~joe/thesis/armstrong_thesis_2003.pdf

It is a well written text on building reliable distributed systems in general, and Erlang in particular.

I don't think it's useful to conflate distributed computation with concurrency, though. Erlang's model is designed for distributed computation, but as a side effect can be applied to concurrency. Go and Clojure are designed more for single-host concurrency. As long as networks are significantly slower than motherboards, there's always going to be cause to treat these scenarios differently.

The awesome thing is that for Erlang it doesn't matter. The two are the same. Here how to send a message a local process on the same node:

    Pid ! Msg.

And here is how to send a message to a Pid located on some server farm in Japan:

    Pid ! Msg.

In general CPUs are probably not going to get faster. Networks might, both on LANs and WAN. There is the 100Gb Ethernet, optical interconnects I think will become cheaper.

On the motherboard # of cores will be increases but at some point, while we get the hang of concurrency and restructure our codebase, we'll quickly get hungry for more cores and memory than a single server can support.

What happens then Erlang program won't have to be restructured when that happens because it provides a suitable abstraction. At the lowest level it will still be Pid ! Msg. That is the beauty of it.

Yes, I know. And from an abstraction level, it's really cool that it works that way. But when you are operating on the same node, asynchronous message passing is sometimes an awkward and less performant solution. It's strictly weaker than synchronous message passing, which means you never have any assurance that the other process received your message. Message passing can also be less performant than properly protected usage of shared memory, which transactional memory allows.

Erlang's popularity for concurrent programming probably comes down to the fact that anything is better than manually managed locks and until recently there weren't any widely-used industrial languages that used synchronous message passing or transactional memory. I don't think it's any coincidence, though, that the designers of more recent concurrent languages like Go and Clojure have not followed the Erlang model.

Well I think in the end its the same in any case. If you use actors or stm in the end you will have to know if it runs distributed or not. I think nobody clames that every Erlang programm makes for a good distributet system.

STM or Actors can both be useful as concurrency and distributed programmes but non is a silver bullet in either or both of them.