His "true parallelism" means two (or more) actual simultaneous executions (threads, processes, etc.) occurring at the same time (redundant with simultaneous, for emphasis). This is contrasted with concurrency (see Pike's talk Concurrency is not Parallelism [0] for an example of how this line is drawn), which may or may not be implemented with "true parallelism". Coroutines, the example Gosling gave, often do not actually have simultaneous execution.

That Erlang forces some synchronization that other languages may not does not affect the ability to (with versions of Erlang since, what, 2007 or 2008?) make use of SMP for simultaneous execution of processes on separate cores or processors.

See also OpenMPI for an example of a distributed programming system that also can't share memory without copying it. Though in its case it was across physical nodes (and so the need to communicate was more obvious, they physically didn't share memory anyways). Still true parallelism.

[0] https://blog.golang.org/waza-talk

And while MPI was designed for multi nodes, I’ve found that in many cases where a code base supports both MPI and openMP (shared memory with multiple threads in one process), the MPI version runs faster because each single threaded process knows that it owns it’s memory and there’s no need for synchronization.

Being explicit about the data being passed around while defaulting to “not shared” has benefits for making use of all your cores more efficiently.

Because anything described as "true X" is a fallacy and everyone has different views on what "true" is, what "parallelism" is and what "true parallelism" is. Sure, if you define "true parallelism" as "share memory between cores without copying it", then Erlang doesn't give you that.

It seems that James Goslings view of "true parallelism" is "let you take advantage of multiple processors", so with that frame of mind, Erlang does give you "true parallelism".

> Coroutines kind of magically sidestep some of the naughty issues in true parallelism. And for me, one of the problems with coroutines, which is why I haven't used them in a long time, is that they don't actually let you do or let you take advantage of multiple processors. You can't do true parallelism.

> But as soon as you've got one of these coroutine-based languages and you try to exploit multiple processors, if you're doing a lot of coroutine-type operations and you don't have enough processors, you're just saturating one processor.

I'm not sure there's too much debate on what parallelism is (simultaneous execution of 2 or more processes/threads). The broader debate seems to be on what concurrency is. Is it a design feature (in which case coroutines in a single threaded program offer concurrency) or is it an execution model (in which case it intersects with or is the same as the common definition of parallelism).

The other debate is on how parallelism (and concurrency) should be done. Shared-nothing or shared-memory? It'd be foolish to say that only one of them is parallelism, but it would be reasonable to debate the merits of each approach in different systems and circumstances.

Thanks, I agree with you overall. My comment was more focused on the (old and "well-used") adage of "X is not true Y" that gets used over and over again to look down on X, while the definition of anything "true" is most likely just personal opinion and is different depending on who you ask.

> The other debate is on how parallelism (and concurrency) should be done. Shared-nothing or shared-memory?

Obviously it should be "shared nothing" as it would allow you to horizontally scale your application and code will be much easier to develop and test. But in reality you inevitably need to have a shared state in one way or another. You could try to move that state into some dedicated service, like database, but it's not always performant enough (or even applicable). So you end up with shared memory because of performance.

I think Rust shows the advantages of shared nothing can be mitigated through novel language design.

I'm trying to name the version of parallelism that can share memory between threads, and so far I have tried "Joint Parallelism" and now lately "Atomic Parallelism" without much success.

What would you suggest?

I'm not sure it needs a special name. It's the default model that almost everyone learns first, shared-memory parallelism/concurrency. It's the whole reason that semaphores and mutexes were developed (to avoid contention for the same resource or resource set from multiple threads of execution).

Yes, but I also mean that parallelism to be atomic/non-locking. afaik only Java/C# can do this on a high-level (without going back to atomic arrays in C) but I still can't explain why, the best I found so far is:

"Many lock-free structures offer atomic-free read paths, notably concurrent containers in garbage collected languages, such as ConcurrentHashMap in Java. Languages without garbage collection have fewer straightforward options, mostly because safe memory reclamation is a hard problem..." - Travis Downs https://travisdowns.github.io/blog/2020/07/06/concurrency-co...

But it seems there is also some benefit to have a VM in addition to a GC for this joint/atomic parallelism.

I wrote that. The problem I was getting at is that it's fairly easy to write various types of lock-free structures in native langues, but you almost always run into the "memory reclamation" problem.

E.g., you can look up something like a lock-free linked lists, the first of which were described decades ago. They have an algorithm to delete a node from the list, but what often isn't emphasized is that you can't just free that note by calling `free()` on it, because you don't know if any other readers might happen to have a reference to that node, and thus may access the underlying memory. If you free it, they might access freed memory.

In general, this is a hard problem to solve. You can wait, but how long is long enough?

Garbage collected languages solve this more or less automatically: there is no explicit free() but instead the GC determines when there are no more live objects and frees it for you.

This isn't unsolvable in native languages, but none of the solutions are as clean and tradeoff-free as you'd find in a GC'd language.

Look up MISD. I think that’s the idea you’re getting at, there really isn’t a lot of hardware support for it which is why for more complicated structures you’re pushed towards higher level languages.

Historically, this is known as "multiprogramming". Multiple programs making forward progress inside the same computer, via interleaved or concurrent execution, the latter being called "multiprocessing".