That said, there are occasions when channels have proven invaluable. I think the real issue is they were branded as a killer feature but in reality their usefulness is a little more niche than a mutex.
Agree - and many people are surprised to learn both: a) how fast mutex ops are in practice, and b) how channels use mutexes under the covers.
Fortunately one can code a replacement for channels in Go using mutexes and semaphores. And that exercise in turn allows one to see that in many cases the channels are just a bad model. Things like having a single polymorphic priority message queue per go routine suits many cases better than dealing with multiple channels.
And this has been known since nineties if not eighties when Ada had to add mutexes after early designs based purely on channels. So it is puzzling why Go made channels such a central feature when the priority queue just works judging by Erlang or many successful C/C++ libraries.
To each their own though. I hear a lot of people prefer it.
The close is built into the language itself which is useful, along with the other built in language functionality. I wrote about it here, with an example of how to do it https://boyter.org/posts/file-read-challange/
I agree that the ergonomics of Go channels leave a lot to be desired. Also, channels are still a primitive: you still need to build abstractions to do anything nontrivially useful with them. I think something like a good coroutine abstraction, or thread nursery abstraction would be ideal.
I think your frustration comes from you wanted to use channels for any scenarios. This is not recommended too. https://github.com/golang/go/wiki/MutexOrChannel
In general (not just Go!), don't send tiny amounts of work across any sort of internal boundary (goroutine, thread, spark, green thread, whatever they call it). The amount of work you send should significantly exceed the costs of sending it. Follow this advice and the perf differences between mutexes and channels will almost certainly not be relevant; fail to follow it and neither of them will be fast enough. I don't think there's actually a lot of programs in the wild where the performance difference between the two would be the make-or-break difference. Non-zero, but not many. In practice it's pretty clear developers will spend much more time worrying over it than is justified.
(A common benchmark I see new programmers apply to any language that claims to be good at multithreading is to try to "parallelize" the act of adding a few million integers together by sending individual integers or addition problems out to threads, and wondering why it's 10-100 times slower even though all my CPUs are at 100%, so why does multithreading suck so hard in this language? The problem is that no matter how cheap the send operation is, it's going to be dozens or hundreds of operations, whereas a single integer addition is generally a single cycle, or possibly, amortized to less than that depending on how good your compiler is. You just need to be sure to send units of work larger than the cost of sending them, which is generally not that hard. This isn't just Go, I've seen this charge leveled at Haskell, Erlang, and Python's multiprocessing too, and I'm sure others have seen it in their own communities. It's a very common mistake.)
Jerf, you did it again! This is gold! Succinct and to the point.
I'm going to think on this one.
When you get out into discussing network matters, it gets a lot fuzzier. You may "know" that you're only .025ms away from something in some specific system, but your TCP-based API doesn't necessarily "know" that and will happily use the exact same code to do something that takes 4 orders of magnitude larger. That's enough difference to be problematic.
https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-s...
The proper question to ask is whether your application (or your programmer) lends itself with the CSP paradigm or not, and whether this allows you to write simpler, more robust code.
Extremely performance sensitive code will always need to be written In lower level paradigms, using e.g. atomics directly - this doesn't matter for most code though. And to be frank, extremely performance sensitive code is past the capability of Go for quite a few reasons.
For anything performance sensitive, the only thing that matters is to profile your code.
combined with goroutines, channels are absolutely one of the best features of this language, to me.
async & await, on the other hand, still confuse me and trip me up today, even after using them for years, because of very weird implementation minutia and edge cases that I always seem to stumble into.
I wish async and await would disappear from the face of the earth and I wish coroutines and channels were in every language I use.
It really only means that CSP looks nice on paper but (at least as implemented in Go) doesn't work out well in practice. There's lots to like about the language even if the CSP theory didn't pan out.
When would that pattern not be useful?
* Use `reflect.Value.Select`[1]. Having to reach for reflect feels ugly for such a common case, and the performance of the reflect-select is much lower than the native select.
* Create a single channel owned by the reader, pass it to each writer, and arrange for this channel to be closed when the final writer exits, through a waitgroup. There is an example under "Parallel digestion" in the Go Concurrency Patterns blog post[2]. Note the little details to get right. We must launch a separate goroutine to monitor the waitgroup / channel closure. If we accidentally do it in-line at the wrong level, everything will work fine if the total number of items written to `c` is less than `c`'s capacity, but will hang once a worker becomes blocked on `c`. Additionally, the waitgroup is threaded directly into the writers, which may be more difficult if those are implemented in some other generic package.
* Wrap the above pattern up into a `merge` function, such as the one under "Fan-in, Fan-out" in the Go Concurrency Patterns post[2]. The lack of generics means we will have to copy-paste this function everywhere we want to use it. Additionally, this launches a goroutine for every channel being watched, which strikes people as "expensive" for such a simple operation.
* We can construct a function that takes two channels and launches a goroutine that selects between the two and writes to a merged output channel. By constructing a tree of these we can merge an arbitrary number of channels. This is really just an optimization of the above.
None of these options are particularly intuitive. Too often I've instead seen developers create a single channel owned by the reader and either:
* Assume it is never closed and the reader doesn't terminate until the application does
* Rely on some external mechanism to know when to stop reading. If the reader can stop reading without confirming that the writers have stopped writing, this can lead to the writers becoming blocked on sending into this channel, which may prevent them from performing necessary cleanup actions (signaling `.Done()` on a waitgroup, for instance) that cause hangs in other areas.
* Thread a cancellation ctx through every reader and writer. This ensures that nothing hangs, but can result in messages that are sitting in the the channels being dropped. If other areas of code have an assumption like, "every accepted request will receive a response", this can break that.
In addition, many developers have a gut instinct to add some amount of buffering to their channels, which usually results in these backpressure / channel issues being papered over during low-load unit tests, only to rear their head during higher load integration tests or in production, when the debugging story is much more difficult.
Never really a good idea, and never necessary.
> create a single channel owned by the reader
Channels cannot be effectively owned by their reader(s), the contortions you have to bend the code into to make that work never really make sense. That's just a constraint of the type, but it's hardly a problem -- it makes the thing easier to model. So this isn't really an option on the table.
> a Merge function
Yes! The answer. And goroutine per channel is kind of the point of using them! Nothing inefficient about it.
> a function that takes two channels . . .
Now there's some inefficiency! ;) No reason to do this, given Merge.
--
> None of these options are particularly intuitive.
The merge option seems perfectly intuitive to me, assuming you understand channels have to be owned by a singular writer.
With channels in Rust, the channel is closed when either all senders or all receivers are dropped. This means that doing the default obvious thing is also correct, for a much larger set of tasks than made easy by Go's API choices, and it stays correct under refactoring.
Wild!
The rule is that channels are owned by a single goroutine, who's uniquely responsible for sending on them, and closing them. That's basically it. Do that and everything works fine, in my experience.
Go's implementation of channels are very simple to use for some very simple use-cases, but Go's API choices mean there are a lot of subtle, non-obvious details you need to learn and keep in mind to do anything nontrivial.
I really like https://medium.com/justforfunc/why-are-there-nil-channels-in... as an example. Reading from two channels safely should be a simple task, but just doing the intuitive thing will look like it works for many uses until it starts fabricating zero values, or blocks forever, or spins the CPU at 100% doing no work.
I really love channels, but I really hate working with Go's channels.
The channel API in that other language well-known for its good concurrency support is much simpler to learn and use for me, without as many subtle sharp edges, but that's possibly a bit off-topic, so I've removed a detailed comparison.
I was specifically trying to explain why I say that Go's channels are not intuitive, because they require studying and memorizing these other arbitrary complications.
I'm also curious about what docs you're referring to, exactly. Here's what I found when looking for golang channel docs:
https://golang.org/ref/spec#Channel_types
If the documentation's described behaviour, along with code patterns to accommodate that behaviour, are intuitive to you after reading this, then you have a very different perspective on the world than I do.
I also found these: https://tour.golang.org/concurrency/2 https://golang.org/doc/effective_go#channels
Unless I've missed it in my reading, I don't see any of these clearly stating that the single-return form of channel receive will fabricate zero values when misused, or describing how you need to replace a closed channel with a nil when selecting on multiple channels to avoid spinning the CPU when it's been closed.
I agree that this stuff is learnable. I have learned it, and so have you. I agree that there are learning resources out there that help with learning the nuances of using Go's channels well.
Hopefully this can help you feel less shocked the next time someone says that Go's channels are not intuitive. If you disagree, can you explain more about how Go's channel management choices are more intuitive than the alternatives to you?
[Edit: I found the documentation on producing a zero value when reading from a closed channel here: https://golang.org/ref/spec#Receive_operator]
You certainly missed this: https://golang.org/ref/spec#Close
You might also need read https://blog.golang.org/pipelines and https://blog.golang.org/concurrency-timeouts
If you need a good summary, you could read my articles: * https://go101.org/article/channel.html * https://go101.org/article/channel-use-cases.html * https://go101.org/article/channel-closing.html
I keep going back to that page whenever I need it, because I could never remember what the axioms are.
Thanks like fanouts, multidispatch, sinks, chaining, simple concurrency, etc will likely emerge in a lib in the future.
I use mutexes.
Yes, channels are not always best solutions for any case. There are cases mutexes are more suitable. Just choose the best solution for specified cases. Always sticking to one is not a good attitude.
We used to do a coding exercise at a previous job that was really well suited to mutexes and whenever someone tried to use channels they ended up with a much more complex solution.
Equally, there are lots of cases which lend themselves to channels really well. Waiting on multiple things at once is hard to without channels and select for example.
Yes, with predictable results.
> it can be closed
Yes, by the owner/sender, who then understands it as invalid. The effect of receiving on a closed channel is well-defined and predictable.
> and there's different behavior of [recv/send] whether it's in a select or not
I don't think that's true? What do you mean?
> And you often end up with multiple channels as well
Sure! That's part of their power.
> Reviewing code with channels is often more confusing than code that uses mutexes
I totally agree that goroutines and channels are often over-used by new Go programmers. If what you're trying to do is simply protect concurrent access to some shared state, a mutex is the far better choice.
Furthermore, you can easily have hierarchies of "goroutines", everything has to be painstakingly built from scratch and is very error prone.
> Another aspect of Go being a useful programming environment is having well-defined semantics for the most common programming mistakes, ... Quoting Tony Hoare
One of the most common programming mistakes of all, dubbed by Hoare himself as the "billion dollar mistake": null pointers, yet it was put into go without any consideration.
I get the distinct impression that people who invoke this boilerplate Hoare quote argument don't understand that null pointer bugs are "a billion dollar mistake" because nullable pointers have been idiomatic in every major programming language for the last 40+ years, not because they are individually particularly expensive.
Yeah, Rust-like enums would probably be strictly better than null pointers, but people talk about null pointers like they are going to doom your project when in reality they're mere papercuts--a small subset of all type errors--and there are whole projects that are written in fully dynamically typed languages! Indeed, there are worse problems than type errors in a language--you could have an impoverished ecosystem, poor tools (especially build tools), or abysmal performance, and many of the most popular programming languages have two of the three and null pointers. Go is fortunate to have only null pointers (papercuts) working against it.
There's no difference in what side closes the channel, I think, just in how a closed channel behaves when you try to send or receive on it.
Personally, I think it's terrifying that the "easy" option just lies to you instead of crashing the program, but I kind of see how it fits with some of Go's other design decisions that I'd also have made differently (Option/Result instead of pervasive nullability).
If there was supported for optional value types rather than using pointers that would help too.
But the vast majority of Go code doesn’t use channels. The fact that lots of people have trouble with them doesn’t change the fact that the vast majority of Go code does follow this principle.
> Programs that modify data being simultaneously accessed by multiple goroutines must serialize such access.
> To serialize access, protect the data with channel operations or other synchronization primitives such as those in the sync and sync/atomic packages.
Go gives us `sync.WaitGroup` and `errgroup.Group` as abstractions to manage concurrent operations. I always end up guiding developers towards using these unless they absolutely can't (like responding to OS signals requires interacting with channels).
Anymore, I only use them as a semaphore to keep less than X routines running at once.
but this means there are as you say subtle non-local effects...and these kick in exactly when you don't want them - when you are trying to thread in a new feature in an existing codebase.
I prefer to just ignore channels. then that statement makes sense :)
This explains it (tl;dr nobody bothers to formally define what errors they return, and the blindly propagate such unspecified errors, and it leads to ambiguity and unintentional program behavior)
https://www.lainchan.org/%CE%BB/src/1612397614615.jpg
- wrapping every underlying error in your own error type is not helpful
- but defining every underlying error in your own type confuses your implementation (and especially dependencies) with your interface
- and most errors can't be handled in code anyway
There are a few errors (mostly in file operations) that are individually handled as normal parts of life, but otherwise they really are just there to send back up to the user. It's more important to know where an error can happen than what it is.
Other than that, I can't think of any way the error handling subtly introduces bugs.
If you index into a map, then you get a tuple (val, err), where `err` is non nil if the value exists, and `val` is the value in the map or "the default construction of the value".
Go also silently discards tuple members - e.g. `v = map[4]` is valid, and silently discards the error.
This kind of means that all go maps can accidentally be defaultdicts (in Python terminology), but it's also an excellent way to introduce bugs without meaning to.
The mistake of the programmer missing an error that should be handled in a particular situation could be considered a subtle bug, I suppose. This is different to languages that have compiler-enforced error handling, where if you forget to handle a certain case the code won't compile.
(The only language I know where this is nominally "solved" is Java if you confine yourself to its checked exceptions, but nobody considers that solution "satisfactory", and indeed can be taken as a rough-and-ready proof that there may not even be a "static" solution to this problem that is usable. I'm beginning to think of it as "Errors don't compose, because they contain every possible pathology that would prevent values from composing." At this point, based on the consistent failures trying to create composable error solutions despite substantial effort poured into it across multiple languages, I am going to operate under the assumption there is no such solution until someone produces one.)
..at most once!
I find this philosophy strange when they talk about the communication primitives. Channels are extremely subtle in my experience. A channel can be buffered or unbuffered, it can be closed, and there’s different behavior of read/write whether it’s in a select or not. And then you often end up with multiple channels as well. Reviewing code with channels is often more confusing than code that uses mutexes in my opinion.