I have not once had to use sync/atomic and I wrote and maintain a massive data aggregation and quantitative algorithm platform 100% in Go at work.

Channels in Go are not a function, they are a primitive type.

In contrast to semaphores aka. mutexes, channels are highly recommended since, when used correctly, they can serialize concurrent access very efficiently. Take care of how to use them - do not pass tiny amounts of work over channels, pass around a chunk of work and work in batches.

Normally the work you will do per item will greatly exceed the 90–250 ns it takes to move the item through the channel, so it’s just not worth worrying about.

Channels are slower than copy() for just shifting bytes, but in a simple scenario are about as fast as a naive self-made channel implementation using the sync package.

The choice of channels vs. mutexes is one of design, not implementation. BOTH use LOCK XCHG and pay the price.

Also, channels are blocking data structures. APIs should be designed synchronously, and the callers should orchestrate concurrency if they choose. If you just want to synchronize access to shared memory then use a mutex. These are not good use-cases for channels to begin with.

As they say: Share memory by communicating, rather than communicating by sharing memory.

An easy example of this: Use a channel with a buffer of 1 to store the current number, fetch it from the channel when you need it, change it at will, then put it back for others to use.

That last example, where you store a single number in a channel, seems to contradict your recommendation to "not pass tiny amounts of work over channels?"

It's supposed to be an easy example of the concept of sharing memory by communicating and has nothing to do with the recommendation to not pass tiny amounts of tasks.

Interestingly, even though Go is most popular for writing web services and promotes channels, we almost never use channels in our microservices written in Go because they are not persistable. If you care about not losing your data on power loss or a panic, your work queues have to be persistable and retriable. Channels IIRC are just in-memory queues with mutexes under the hood and provide nothing of that. We only use them for a few tricks/hacks like listening to signals to safely stop goroutines.

That makes sense: channels are a way of organizing concurrent programs, not a way of organizing distributed systems. They're a tool you might use to build a persistent work queue (or, like many programmers, including people working on the Go stdlib, you might use other synchronization constructions), not a work queue themselves.

When I first learned Go, my impression (and also tutorials usually imply that) was that they are a nice elegant way to build concurrent pipelines where goroutine A sends work items to goroutine B which can, in its turn, send some work items to goroutine C etc. But in practice for robustness we prefer persistent queues for such pipelines because on power loss, a panic, or if your application is simply being killed on redeploy, your program can lose data or end up in an inconsistent state, because whatever was the in the channels is completely lost (and the work is already half done). So it leaves us with only a few use cases where they're really useful such as basic goroutine coordination. I think what you are saying is that they're a synchronization primitive akin to mutexes, atomics etc., no more no less, and I'm fine with that, but then it's not clear why then channels have a special syntax and why they are sold as one of Go's strong points, if it's just a niche synchronization primitive. The only useful case I found for them was to send an empty struct to a goroutine to signalize that there's a new work item in the persistent queue, to avoid having to poll the external queue too often. I wonder if other web devs have experience similar to ours, or maybe there are other use cases for channels we are not aware of.

The point of channels is to have communication between threads that's easier to reason about than explicit locks. Rather than, for instance, unlocking a map to update it, you treat a single thread as a "server" for that map. If you don't want to structure your programs that way, you'd just use mutexes.

In older web languages, like PHP, I would reckon it’s like making a self-http call where you don’t care about the result and just want to trigger some work elsewhere in the application async. I guess with Go channels, you’d lose out on concurrency, automatic retries (depending on infra), and debuggability — you could do the same with Go and http calls though and just drop the channels completely.

But yeah, sometimes things break. We rely on things like nginx options to retry GET requests and idempotency in the design; failing gracefully (via a shutdown callback to always return something to a caller); ensuring work is completed before writing a successful response anywhere, along with being idempotent; and, ensuring there’s observability in every long-running task.

> That makes sense: channels are a way of organizing concurrent programs, not a way of organizing distributed systems.

Exactly.

Nit: sync/atomic is not undocumented: https://pkg.go.dev/sync/[email protected]

What makes you think sync/atomic is undocumented? https://pkg.go.dev/sync/atomic

The documentation says

> Package atomic provides low-level atomic memory primitives useful for implementing synchronization algorithms.

> These functions require great care to be used correctly. Except for special, low-level applications, synchronization is better done with channels or the facilities of the sync package. Share memory by communicating; don't communicate by sharing memory.

There was a long mailing list discussion discussing use cases and certain desired behaviours that the people on the list agreed upon. In the end they chose not to make those behaviours explicit and leave it somewhat open-ended and instead put in the phrase "great care" to cover it off, rather than define it.

Do you have a link? Curious about the use cases

I can't find the original thread that had a specific example of a sequence of instructions where the desired/actual result wouldn't be guaranteed by the documented behaviours, but was decided to leave it as-is. The closest I can find[0] is lacking specifics, though does say somewhere that better docs would be nice. And the 'great care' warning/disclaimer is still in the docs.

I found it while using sync.atomic to make some implementation faster, then wanted to know exactly what the language/runtime behaviour/guarantees were and was surprised but not shocked to find the 'for the go authors but not thee' attitude of the conclusion. It did have some reasonable founding where they mentioned that by formally defining it, it could be over-specified precluding future optimizations on even newer processors/architectures.

I would have preferred if the documentation explicitly listed the parts that are not fully defined with reasons for doing so given. That's better than a generic danger signpost. The irony here is that Go was supposed to be not-Java but has the same 'for mainstream' mindset rather than being the sharp tools for the sharp developers.

[0] https://groups.google.com/g/golang-nuts/c/0uPDCRiBWqc

I was gonna answer:

"Channel ops are wait free if the buffer has extra space though, so that should be fast"

But, I looked it up and channels indeed use locks even in case of buffered ops. Sigh. I guess MPMC with scheduler yield isn't the easiest thing to write and maintain.

Unless you're using something like an RCU queue, you're still going to have to lock operations around moving the queue pointers, or you could have producers writing over each other or writing into a slot a consumer has already read from.

I don’t think channels were ever billed as peak performance abstractions. I certainly don’t use them as often as I use mutexes or atomically for parallel code, but frankly I rarely write parallel code in the first place because it’s harder to maintain and I rarely need the performance (single threaded Go is really performant already compared to other languages in its class).

Heh. For the past 3-ish years I operate a flock of microservices in Go. I don't really write the code, but do quite a bit of reading to debug and fix stuff. This month I've seen a channel in production code. I did a double take and took my time to reflect on the language.