If you say so. I can't count the dollars I've made fixing other people's poorly written multithreaded code in my entire career, including in the last 3 years.

Thread support is [more or less] standardised in all major OS-es now, sure. Doesn't change the fact that it's an extremely bad fit for the human brain to think about parallelism.

Stuff like actors with a message inbox (Erlang/Elixir's preemptive green threads) or parallel iterators transparently multiplexing work on all CPU cores (Rust's `rayon` comes to mind) are much better abstractions for us the poor humans to think in terms with. Golang's goroutines are... okay. Far from amazing. Still a big improvement over multithreaded code as you pointed out though, I fully agree with that.

I might be projecting here and please accept my sincere apologies if so, but it seems to me you are a bit elitistic in your comment. Multithreaded programming is still one of the most problematic activities even for senior programmers, to this day. Multithreading bugs get written and fixed every day.

At this point I believe we should just move to hardware-enabled preemptive actors where message passing is optimised on the hardware level, and just end all parallelism disputes forever since they are an eternal distraction. (The overhead when utilising message passing today is of course not acceptable in no small amount of projects. Hence the hardware suggestion.)

I'm curious why you feel this way? Certainly anything with multiple (more than one) thread is a lot harder than single threaded. Given.

But I think the human experience is rich with real world counterparts that can make multithreaded programming "natural" (not necessarily easy, there's a difference). Basically any process in real life you do that involves collaborating with multiple people is a collaborative multi-threaded process. When forced to grapple with multi-threaded solutions, I often ask myself, "if I had a room full of people working on this problem, what would have to be in place to make the operation flow smoothly?" For me, this anthropomorophisation of the process makes it a "good fit" for how my brain is used to solving lots of real world problems.

On the flip side, I haven't had a lot of luck finding real world experiences that I can model coroutines/async/etc on. So they may be ultimately easier, but if our rubrik for "fit for the human brain to think about" is what the wealth of human experience has evolved our brain to handle well, I'm less convinced.

But I like learning new things. Maybe I just haven't seen the lightbulb yet. Help me see your point of view?

I feel we might be having too incompatible perceptions of the world but I'll try.

Every communication (human or otherwise) is exactly actors with message inboxes. We the humans can't react to 3 incoming conversations at the same time. Our brain queues up what it hears or reads and then responds serially, one by one (not necessarily in order of arrival). Basically an actor with a message inbox, like in Erlang/Elixir.

The way you assert multithreading mimics the real world to me is not convincing. One possible example in favour of your point of view I could think of is probably a table full of food and 20 people reaching and grabbing from it at the same time. But even then, two people cannot successfully get the same piece of meat. So not a true multithreading in the sense of N readers/writers competing for the same resource; it's more like a shared memory area where parallel writes are okay as long as all writers reference different parts of it.

Sure, a lot of stuff collides in real time without synchronisation out there, that much is true. But you likely noticed we the people don't cope with the Universe's chaos very well and rarely manage to grasp it properly. So is it really a good way to model deterministic and non-chaotic systems with programming? To me it's not.

I am curious why do you even think multithreading mimics the real world at all. Elaborate, if you are willing? (We might think of different definitions of multithreading in this instance.)

That would be actors or processes communicating with each other, not a collaborative multi-threaded process.

Real world has pretty much no natural notion of threads, only of actors.

The closest s/w equivalent I could think of is Unix pipes. When I do “cat foo.csv | sort | uniq -c”, cat/sort/unique aren’t coordinating with each other, each takes the content from stdin, transforms and place it in stdout.

The assembly line principle can also be applied to the concept of threads. Workers collaborate by handing items off to one or more workers(threads) down the line. Thread pools are just micromanagers constantly reassigning the poor workers, but at least the make a lot of friens collaborating.

Not the OP, but in my experience most devs are bad enough dealing with state and flow in single-threaded code already. Including myself. But I've at least accepted that and try to mitigate it by avoiding things that make it worse (like threads). Meanwhile, I watch people around me doing code by spaghetti (throwing it on a wall and seeing if it sticks). The better devs don't apply that to their code, but still end up applying it to their "reasoning" about a problem domain, especially a business one.

E.g.: "Screw it, if this variable that I need is null I'll just return a False in this function I'm writing" ... without ever thinking "why" this variable might be null and how that explanation impacts the problem they're solving. Nevermind the fact that they're fully OK with propagating such a broken state further into the program with their False response. Now some poor other dev has to figure out why this function is returning a false result. Odds are they'll do the same thing and now we've doubled the amount of bad states.

Adding in even more complexity in terms of being able to "predict" code state by adding threads is a nightmare. I would most certainly protest and heavily advise against anyone using threads unless it's really really the thing that's needed to solve the problem. Otherwise, it's just adding more a lot more complexity for very little benefit if any at all.

I would argue that the code equivalent of people collaborating in the real world is multiple processes with IPC (or even further afield, multiple nodes in a distributed system), not threads. You don’t share memory with other human beings. Imagine how productive you could be if you did!

(Technically these aren’t best expressed as SHM but rather as a tuplespace or greedy worker-pool abstraction, but they can be achieved starting off with “just” an SHM threading primitive, so they count, I think.)

This really depends on where you draw the boundaries. In a computer everything is "memory" of one kind or another, but that doesn't necessarily correspond to human memory. The analogy works if you think of the "actor" as merely the thread's context and dedicated working space (stack), with everything else being part of the surrounding environment. In the real world we don't have anything akin to process separation, except by convention. Even the body itself can be directly manipulated by others. The convention we use for communication between individuals looks more like a message-passing system, but physical manipulation of matter is more like operating on data in shared memory. There is no physical law preventing someone else from coming along and messing with an object I happen to be holding any more than the rules of a computer prevent one thread from messing with a data object held by another thread.

Async, actor-like processes are everywhere in the real world - "something comes into my inbox, I do my piece of work on it, then send it to their inbox". Or "I'll send this off, do something else for now, and then go back to working on that when I get their reply".

As a programming model, threads don't really help you to achieve this. When there's a bug, your only option is pretty much to think very hard and try to figure out what happened. (Or to use gdb with scheduler locking, which is about as bad.)

In short if it's not a good fit for your brain, change your brain!

I am curious as to how your h/w experience makes it easier hacking on the linux kernel....

Go is sort of like saying "we're not going to deal with parallelism directly, just throw everything into fifos to talk to other places" mostly you can't afford to waste gates like that in real hardware (there are places where it makes sense - I've built a lot of display controllers).

I think the big difference is the way you end up thinking about time and timing holes in code (dealing with interrupts), after building gates for a while i think you tend to see the timing holes more easily

Not sure what you mean by your statement that hardware engineers are trained to think differently about it. I am not sure there's much overlap between HW and SW engineers in terms of parallelism modelling. Do you have a few examples?

However having said that I find myself designing silicon again, change is good for the brain I guess

Hardware people tend to think in terms of netlists and pipelining so they have to worry about parallism on every clock

Threads in C are hard if you have a sloppy codebase with global variables everywhere and a general lack of structure and abstraction. Unfortunately, this describes many many C codebases.

But stopping to use C is a good start (for whoever has the choice)!

Not as toy, proof-of-concepts. Fully fledged replacements for all that stuff that can be used in our our daily work. Build distros of the stuff so that we don't have to pick and choose this stuff and replace the bits of our systems in a piecemeal fashion.

They could show us how it's done instead of talking endlessly about it in online forums. So let's face it, it's never going to happen.

As for managed code, it's being used with huge success in a lot of places (not in OS-es or drivers, yes) but that's quite the huge topic by itself.

I'm not defending C, I'm merely sneering at its noisy detractors who spend more time complaining about it than supplanting it.

Perhaps as the Rust community grows in size and momentum this will happen more. Right now it seems like a bit of a manpower problem.

Note the section 1.7 in your own link ("End of the Lisp machines"). The Lisp software ecosystem was never as rich, broad and varied as the C/C++ software ecosystem is today, before it died.

BTW, I only stumbled across your comment by accident, since you replied to the parent poster instead of me. Parenthesis mismatch?

Cheers.

It's perfectly fine to continue using an existing C codebase for a program, not exposed to the public internet, that's maintained by a focused group of maintainers. But on the other side of this spectrum, for large exposed projects like OpenSSL, Chromium, or even Linux, C/C++ has become risky.

My argument at least is that, given the choice, there exist much more productive ways to make your employer money than to try and learn nuclear phys... I mean multithreaded programming. :-)

I am not arguing in favor (or against) threads. I am not contesting that they lead to more bugs or not. What I was contesting was the assertion was that "threads are not a natural fit." My point is that procedural programming (which nearly all programming is at least small level, minus languages like Prolog) is something human beings have been doing for thousands of years. Look at "recipe" for doing something, and you have procedural program. Look at any recipe for lots of people working together to do something, and you have a pool of cooperating procedures. It's something we get schooled in our whole lives.

FP makes the programmer rethink the way that the problem is solved. I think that adding threads on non FP code makes it hard from the get go.

Don't hide it on hardware, rethink the approach instead.

As for FP... I remember a guy saying "imperative/OOP shows you how it does stuff, while FP shows you what is it doing". It was a very eye-opening statement for me.

I clearly remember the feeling of just getting it as well. It was a very binary moment :)

    Multithreaded programming is still one of the most problematic activities even for senior programmers

The trouble is we need to stop hunting whales if we're going to move the needle in a permanent and significant way...

* https://en.wikipedia.org/wiki/Amdahl%27s_law

My second favorite class in college was the elective for distributed computing (which I took instead of compilers, but I wish I'd taken both). It felt good to me and I came out into the job market ready to concurrent all the things.

By the time I'd worked with a disparate group of coworkers, I found out that 1) my enthusiasm was a rare thing, 2) that this was with good reason (many, many burnt fingers), and 3) that just because I love something doesn't mean that it should have a central role in a team project, even if I am the lead.

In a similar way that I 'got dumber' by learning to think like a user (one of the downsides of studying UX) I sort learned to reach for concurrency after I'd tried several other options.

... but here I am, years and years later, debugging other people's async/await code for them. Nobody likes to have to have someone else fix their mistake, but it goes a lot better when I start by telling them this shit is hard. Which it really is. Even when (or maybe especially when) I'm confident my stuff is right, I find a few boneheaded mistakes that I have expressly warned others to avoid in stuff I wrote uncomfortably recently.

That being said, developers often have a bit of learning to get there, but that can greatly be accelerated by good team/mentor code review and discussion.

It's just that there exist ways to shoot yourself in the foot quite easily.

Example: I'd find it much more intuitive if writing to a closed channel returned an error value and didn't issue a panic.

But I'm guessing that the Go programmers get used to the assumptions that must be made when working with the language so such things are likely quite fine with them and cause them no grief.

Better with panic than silent errors and flawed logic.

Threads are definitely a valuable tool for performance-oriented work and things like eg; worker pool patterns aren't always a great fit and you end up with more overhead.

I know there exist a group of programmers that really don't have a choice and have to use `pthreads`. But if you do have the choice then IMO sticking to multithreading programming is very backwards and invites a lot of suffering in your future (or that of your future colleagues).

Just use actors with message inboxes. Measure, profile, tweak. If you exhausted all other possibilities and you absolutely positively can't upgrade your server ever then sure, ditch actors and write multithreading code.

Futures and Promises go a long way in making that better. That's not to say that you can always rely on them, but it is far easier to get them right than it is to get threading right with shared mutable state right.

Actors, Channels, Futures, promises, reactive programming, etc. They all have one thing in common. They kill off shared state in favor of message passing.

Yep, exactly.

Everybody can screw up syncing and message passing as well, given enough lack of experience, or schedule pressure, or simply not getting it. But it's much harder to screw up that while conversely, it's extremely easy to screw up multithreaded code with shared state.

More-or-less, but no barriers on OSX (at least I had to hack around that for a user 6 months ago)

"Multithreaded programming is still one of the most problematic activities even for senior programmers, to this day. Multithreading bugs get written and fixed every day."

Strongly disagree. I am going to claim that threads / locking are not hard to write if one has at least a bit of common sense and discipline. Problem is that we have hobbyists without any trace of knowledge on how computers work producing commercial software.

Goes like this: I do not want to deal with memory management - it is too hard, I do not want to think about types - my poor brain can not comprehend those, pointers - OMG what are those beasts, threads and locking - gonna commit Seppuku. I will advise those to come to a logical conclusion: programming is frigging hard when one wants to produce a decent product rather then buggy POS. No matter how many concepts the language hides / converts to something else there will be always something else they will not be able to wrap their mind around. So how about trying gardening instead

You don't have to make this about implying that others don't have "at least a bit of common sense and discipline".

Working with epoll or select/poll is really nasty when dealing with "some data" that's coming on a socket, where as IOCP allows you to tell the kernel "as soon as you see some data destined for my thread, just populate it in the memory here, tell me when something is there and then tell me how much is there"; for high performance C++ programmers this is basically invaluable and as far as I understand there's no direct competitor in UNIX and unix-like land. (although; according to the C++ devs where I work, "kqueues are less braindead than epoll")

And yes, you correctly point out that on nix and Win32, interaction with the kernel is often different. Win32 does a lot right, I won't ever speak negatively of it, it is the desktop OS of choice by a huge margin. It's just that nix and Win32 are so different from each other, that you can't generally write a single implementation of your low level thread interaction for both classes of platforms.

If the thread is waiting on a specific set of objects (e.g. file descriptors, kqueues), then this can be done easily, but it's still not quite as simple as it has been under Windows (and by the way, IOCP is built on top of the underlying mechanism that makes this possible on Windows; they are not the mechanism itself)

io_uring is "different" and definitely has some advantages due to its shared ring buffer. but NT has a core/generic set of async completion/wait/etc functions that can be applied to basically the entire API surface.

I wouldn't assume it does until I've verified and read the code, I know last year at plumbers there was a fuss about the fact that a lot of operations which should have been working wernt. Frequently in odd ways, for example direct read worked but not direct write IIRC.

There is no equivalent in UNIX land to the thread-agnostic asynchronous I/O primitives available on NT. When paired with a robust threadpool API (Vista+), it is an unbeatable platform.

This article is from 1995, and they were out a bit earlier than that. Sun had been shipping 4-core workstations with thread support since 1993.

The SPARCstation 10 came out in 1992 and had two MBus slots. Each slot could take a CPU card with 1 or 2 CPUs on it.

Although SunOS 4.x had very limited multi-CPU support, SunOS 5.x had support for multiple CPUs. SunOS 5.1 (Solaris 2.1) came out in 1992 and supported SMP, and SunOS 5.2 (Solaris 2.2) came out in 1993 and introduced a thread API.

By 1995, they had also introduced the SPARCstation 20 and the 64-bit Ultra 2.

They also had some server systems (like the SPARCcenter 2000) that had a backplane with several boards that could themselves have CPU cards. I don't know the maximum number of CPUs, probably something like 20.

Info from my memory and:

http://mbus.sunhelp.org/index.htm

https://en.wikipedia.org/wiki/SPARCstation_10

https://en.wikipedia.org/wiki/Solaris_(operating_system)

That depends how you define "work". Java has a decent concurrency library such that if you have a limited-scope project implemented by 1 or 2 highly-skilled programmers, you stand a good chance of avoiding thread safety issues. But as the project grows, the probability of thread safety issues approaches 1.

> Event driven programming has also become threaded, this is how the whole reactive family of frameworks, node.js, etc, handle parallelism.

Event-driven programming is way less painful than Java threads. With threads, in every place you mutate anything you have to ask yourself, "what happens if my thread is preempted here?" With event-driven programming you only do that sort of thinking in places where you see the `await` keyword.

Event-driven programming is a different pain in the ass but still a pain in the ass.

The current status of the Web speaks for itself: it's event-driven based JavaScript, and it's full of state/transitions bugs everywhere.

In event driven you trade concurrent r/w access issues for a loosing execution contexts issues and out of order issues. The result is often a mess of spaghetti callback close to unreadable.

The less worst of the world might be simply a proper coroutine system, which seems to come to almost every programming language after 30 years of existence

Thinking about asynchronous behavior is always tricky, and code designed to run on threaded systems isn't generally arbitrary code with locks thrown into it the mix, you strive to write the largest possible reentrant sections, and only lock where you have to, and as infrequently as possible. With reentrant code, it doesn't matter where you get preempted.

Not if you do it right. Interacting with sync-only systems from async systems is a problem, but it's not fair to blame that on async - if you make the whole system work async, you don't have that problem.

I don't disagree with what you've said, I just wanted to chime in that in the context of the conversation to this point the definition is something in the ballpark of "are available cross-platform without jumping through hoops".

in 23 years programming Java (and in the all places where i've worked having been recognized as local expert on threads and synchronization, incl. in the current very large C++ platform project) i have never asked myself that question in the context of the mutation of data.

I don't think I've actually thought about preemption explicitly since the Nintendo64 days when thread priorities figured into our locking strategy. But then we'd also re-implemented the scheduler and, at times, turned off the official OS to go do something we needed to do without interruption. So, like I said, preemption not high on my radar these days, as it falls under the general category of "concurrency". Even an iPhone has 4 CPUs.

The effect is much the same, whether the race is caused by task scheduling, or just because another core got there first.

Modern kernel people still have to worry about locking as well as preemption, if you look at the NT native API's you will see dispatch levels, which control whether kernel threads can be prempted for higher priority activities, linux is similar with the _irqsave() and preempt_ calls.

Is this for example important when syncing with an interrupt handler (no preempt/rt config), which is why there are both spin_lock() and spin_lock_irqsave() which implicitly blocks irq preemption. AKA if you grab a lock that is needed by say an interrupt handler, then you take said interrupt the machine will deadlock because the scheduler won't deschedule the interrupt handler.

This is exactly why I run unit tests concurrently (other than the speed boost), and try to write the tests (and the code) so that it won't break when run concurrently (like inserting something into a DB and then assuming success only if the count has gone up by exactly 1).

I would recommend using specialized tools for the task like Intel's Thread Inspector or Coverity's Helgrind.

The one being a good idea doesn't preclude the other from being a good idea as well. Do them both.

(And also avoid concurrency wherever you can. And of the available forms of concurrency, threads are one of the worse ones.)

There is an impedence mismatch between how code is written (sequential, top-to-bottom, left-to-right text) and how parallel code behaves. That's why people use idioms like actors or coroutines. I think to actually solve this you need a new format for writing code, like a graphical function call graph instead of a text editor.

More than a year later we were trying to port much of the codebase to solaris and it was a nightmare. Pretty much nothing worked right so we ended up bolting on the fork/mmap abstraction on top of much of it and building our own lock wrapper. While we sorta got it working, the solaris port died for internal political reasons and the NT version took off and we never looked back.

NT was designed out of the gate for heavily threaded/async workloads, that stuff wasn't bolted on like pretty much every unix clone in existence.

Might as well forget debugging in promise-land. I think this is partly because they are conceptually problematic as well, apparent once beyond the simplest use-cases, given the number of bloggers that still seek to explain it - and then are forced to edit their post because of errors. And the need to list common mistakes on MDN:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guid...

I don't do intense concurrency in the browser, but I'll do non trivial stuff, like collating responses from a server then sending a request once I have all that information. I think promises are fine for this and it is possible for most programmers to write clean, mostly mistake-free promise based code.

As for difficult to debug? I've never had that issue, neither in JS (browser/node) or C# with the similar but different Task<>.

For example in JS, I can put breakpoints at any point of the promise chain and see what is going on. If they are network requests I'll also look at the network tab. If everything is happening real fast I might use console.log statements, but that is rare.

I think promises are OK for a lot of situations that most of us will encounter developing business software - this might be a reflection on the simplicity of the problems I end up solving. Obviously if you are creating a multi-threaded high-frequency market making trading thing then this might not apply.

The promise pipelining technique (using futures to overcome latency) was invented by Barbara Liskov and Liuba Shrira in 1988.

Yes B. Liskov is the L in "SOLID", if you are wondering!

Happy 40th birthday promises! :-)

I wouldn't be surprised if mathematicians were thinking of this before transistors were invented though.

I think it takes time for things to trickle down. Maybe it takes someone from an academic background (or a curious paper reader) to be forced to use C++ or Javascript for something, but along the lines of "progress relies on unreasonable men/women" they get annoyed and write their own library so they can use the nice Lisp/Haskell/etc. feature they are used to, but often in a more limited way but better than nothing. Then from there it might get hyped to the shithouse, die, or just get used by insiders.

P.S. sorry so late answering your mail!

(I owe you mail too.)

I was terribly remiss not to mention Dojo, a popular JS toolkit which got its promises from Twisted, which of course got them from E, though Twisted modified them a bit. I don't know how it slipped my mind.

I get the impression E promises had a nicer design than JS's for handling errors -- but that's also a vague memory and I never really learned JS's.

Async/await has also mostly replaced bare Promises. The result almost gives you something as usable as threads.

Fwiw I find it amusing the async is promoted as the pinnacle of elegance in concurrent programming when in reality it exists because the JS interpreter wasn't thread safe.

My impression at the time was that developers had a flawed impression of threading complexity and bugginess because they were bolting threads onto existing single-process programs, and their existing hygiene was the problem, nothing inherent to threading.

If you look at the single-process C programs of the era, global variables and global state in general were extremely common. When you start adding threads to such programs to try take advantage of SMP, trying to wrap locks around heaps of unnecessarily shared, poorly encapsulated state, of course you produce a lot of buggy programs. Then everyone starts saying "multi-threading is too hard, not worth it", instead of admitting their programs are a complete mess.

Using modern standards of hygiene, even with threads-naive languages like C and good old Pthreads, I don't find it particularly challenging at all to write threaded programs.

Like you mentioned, I also struggle with the higher level abstractions attempting to make threading easier. Pthreads makes sense to me, I find threads, mutexes, condition variables, rwlocks, all very intuitive and ergonomic to use, but it's probably because I spent a lot of time using that API at a young age.

As I started taking swe jobs in silicon valley in the early-mid 2000s, it surprised me how few people had experience with Pthreads. It blew my mind, four different startups doing C programming with experienced C programmers and nobody was ever familiar enough with Pthreads to quickly review my code without having to rtfm. It's like there was such a stigma surrounding threads being "too hard" a lot of people never even attempted it. But my being a kid learning linux and just excited about new features in my unix, when LinuxThreads arrived and then NPTL, I spent years playing with Pthreads in C and getting my hands on SMP systems just to program them with Pthreads. Those years of playing paid off unexpectedly well in silicon valley, SMP was everywhere and C was still heavily in use on Linux.

I still get a bit of happy nostalgia when the opportunity arises to write some code like:

  pthread_mutex_lock(&foo->lock);
  while (!foo->ready)
          pthread_cond_wait(&foo->cond, &foo->lock);

  /* consume from foo */

  pthread_mutex_unlock(&foo->lock);

I like promises/futures because they tell me when resources will take a while to become available without blocking. I generally don't want to need to know that I need to lock the foo queue before consuming from the queue. I would rather have the queue handle that for me and encode the behavior I want in the type signature:

    next() -> T           sync, blocking
    poll() -> Optional<T> sync, non-blocking
    next() -> Future<T>   async

One thing that i find very convenient in Win32 that is lacking in other platforms (at least using pthreads) is that every thread has its own message queue and you can have threads communicating with each other simply via PostMessage/GetMessage (which also handles sleeping). You can implement something similar over pthreads, but it is nice that the OS provides that out of the box.

Edit: on the other hand Win32 has one really nice feature: you can WaitForMultipleObjects() on essentially anything that has kernel handle, which includes most of IPC primitives. On the other hand this causes the native Win32 IPC to have significant overhead and is the reason why game developers often resort to userspace spin-locks and why Windows 10 had introduced NPTL/Linux-style lightweight futex-based mutexes...

And honestly i never had any bugs with that, if anything i've found it the easiest approach to understand when it comes to inter-thread communication.

It was great fun to write code which drove 8 displays using 8 graphics pipes, with roughly 8 cores working in concert with each pipe. All this work for something that runs faster on the latest iphone using a single thread...

I went on to work at SGI for a few years, and it was still my favorite job ever. It was pure R&D, graphics and realtime systems for their own sake. Today, this doesn't exist. 3D graphics are an applied technology that's part of an app, but not a product research area of its own.

I'm quite certain there are teams here at Microsoft that do just this, and their counterparts in the GPU industry (AMD/Nvidia/Intel - although Nvidia seems to be the dominant one in research)

c.f.: Data Scientists

IRIX also had that weird 'sproc' API that I think was semi-lifted from something in Sequent's DYNIX.