There is also equivalently smugness about the language in general. It is one thing to say there are no undefined behaviour related to the memory model, but given a compiler optimizer, you need to do hard analysis that there cannot be once optimized, with all operations re-ordering that can happen.
The simplicity of the Go memory model, which is vague on purpose, is taken as making it safe. Is it? It mostly mean it is vague, which provide a convenient cover to hide under.
Edit: toward the end, the author talks about adding documentation about compiler optimizations that would be explicitly forbidden. So some of my earlier comments would be addressed.
This really turns me off about Go. And it seems to attract "tech-splainers" that defend the language choices that are made by talking down to people.
Reminds me a lot of the Mac forums where every poor feature in an Apple product is explained with a "let me help you understand why you're thinking about it wrong" kind of answer -- up until Apple fixes the bug / implements the feature to the applause of the same people who were talking down why it would ever need to get fixed.
And just the tone of the article here turns me off in the way it begins with a bunch of quotes and philosophy. I actually agree entirely with “Don't communicate by sharing memory. Share memory by communicating" but I figured that out myself in probably 2003 writing crappy perl scripts that utilized parallelism but I wanted to aggressively avoid all the concurrency pitfalls. Actor models and Erlang later made completely intuitive sense to me. The principle is entirely correct, but its fucking weird that a programming language needs to have a list of "proverbs".
Isn't this just nutpicking[0]? You have that with every language. I can criticize C++ on proggit right now and 3 or 4 people will respond "C++ has every language feature, just use the ones you want/like/etc and ignore the others, your problems are invalid!". Similarly, on an OCaml forum I can find a dozen people who tell me if Jane Street hasn't run into my problem before and solved it, then it's not a real problem and I'm dumb for trying to do it. I can post here or on r/rust (or proggit) and people will tell me that Rust is strictly faster/easier to develop with than any GC language because in the worst case you can always throw `Rc<T>` on everything. I can post on r/python about how hard it is to optimize Python and people will tell me I'm dumb and I can just use multiprocessing, rewrite the slow bits in C/Cython/etc, use numpy/pandas/etc. I can merely register for an account on a Java forum and be berated for my low intelligence. :)
> The principle is entirely correct, but its fucking weird that a programming language needs to have a list of "proverbs".
I don't feel as strongly as you do, but I agree that proverbs and analogies are pretty low quality and invite more confusion than they address. Probably only a rung above analogies and a rung below quotations.
From that article, I'm specifically addressing "Does this movement _promote_ crazies?"
I'd argue the sloganeering attracts them like moths to a flame.
In other words, I agree that "sloganeering" attracts the nuts, but I'm not sure I agree that TFA is sloganeering.
the focus on "simplicity" also is both a good thing, and a great way to shut down any discussion.
If the TFA amounted to "don't communicate by sharing data; share data by communicating <mic drop>" then yeah, I would be on your side, but the author went to the trouble of writing a 4K word essay to support his points. That you and others reduce it to mere slogans isn't a valid criticism of TFA IMHO.
To your point, there are actual people in and outside of the Go community who do this kind of lazy argumentation. For example, someone upthread said (and I'm hardly paraphrasing), "Hoare called nullable pointers a billion dollar mistake and Go has nullable pointers so... <mic drop>".
> the focus on "simplicity" also is both a good thing, and a great way to shut down any discussion.
Pretty sure you could levy the same criticism against the Java folks for "configurability" or the C++ folks for "performance/control" and the Haskell folks for "type safety". It still sounds like you're nutpicking rather than observing something unique to the Go community.
The previous post demonstrated that this is extremely difficult, beyond the ability of cutting edge research for mainstream languages.
> None of the languages have found a way to formally disallow paradoxes like out-of-thin-air values, but all informally disallow them.
The post neglects to mention the Promising Semantics paper of 2017 that resolves the out of thin air problem to (as far as I know) everyone's satisfaction, despite pointing out the previous work that brought up the problem. Similar things are true for ARM's memory model, etc.--this is all stuff that's been mostly resolved within the last few years as proof techniques have caught up with compilers and the hardware. Ironically, the thing that's been hardest to formalize by far in a useful way (outside of C++ consume) is--surprise, surprise--sequentially consistent fences!
It also handwaves away as some unimportant point the reason why compilers provide things like relaxed accesses--it's not just (or even primarily) about the hardware, but about enabling useful compiler optimizations. Even if all hardware switched to sequentially consistent semantics, don't expect languages that aim for top performance to abandon weak memory. And personally, I think it's ironic that at a time when even Intel is struggling to maintain coherent caches and TSO, and modern GPU APIs don't provide sequential consistency at all, people are trying to act like hardware vendors will realize "the error of their ways" and go back to seqcst.
That said, what I have learned from watching this space for a decade is that even formal proofs are no match for the complexity of this general problem. You have to get the definitions right, you have to prove the right statements, and so on. There is still plenty of room for error. And even a correct, verified proof is not really an insight as to why things work.
Experts were telling us in 2009 that OOTA had been resolved to everyone's satisfaction, only to discover that it wasn't, really. Maybe Promising Semantics is the final answer, but maybe not. We need to get to the point where everything is obvious and easy to explain, and we're just not even close to that yet.
Looking at citations of Promising Semantics in Google Scholar I found this Pomsets with Preconditions paper from OOPSLA 2020: https://dl.acm.org/doi/abs/10.1145/3428262. It contains this sentence:
"As confirmed by [Chakraborty and Vafeiadis 2018; Kang et al. 2018], the promising semantics [Kang et al. 2017] and related models [Chakraborty and Vafeiadis 2019; Jagadeesan et al. 2010; Manson et al. 2005] all allow OOTA behaviors of OOTA4."
I take that to mean the jury still seems to be out on excluding all OOTA problems. Maybe the canonical example has been taken care of, but not other variants. And again we don't really understand it.
Undefined behavior is not "data structure being corrupted". When data structures are corrupted, program still functions as written.
Undefined behavior is "things fly off the ceiling", no logic applies and programs start behaving essentially randomly because of the optimizations.
Undefined behavior is not broken data. That is still a well defined behavior.
If you have wild pointer writes (aka arbitrary memory corruption) and almost any non-trivial control flow, the results are pretty much undefined. As in, it is pretty much impossible to specify or reason about what a program will do.
The article:
> ... such races can in turn lead to arbitrary memory corruption.
Tearing writes themselves are not undefined behavior, but use them wrong and you're off into undefined territory, with no way back.
> Undefined behavior is .. because of the optimizations.
Optimizations are popular, but it doesn't matter why the behavior is undefined.
Undefined behaviour would be, if the compiler thinks that it always happens, it (the compiler) can do anything, including e.g. transforming the whole program into
int main() {
return -1;
}
Wouldn't there be an obvious trade off there, requiring an extra heap allocation and indirection every time a slice is used in a data structure? I've always thought of Go as lower-level than Java (since it has explicit pointers, more control over memory layout, etc), which I think makes it a more useful language overall even if not the best fit for every application (since if you do want Java, it already exists and is mature).
In other words, I think both Java's and Go's decisions make sense here.
It's obviously, uh, quite an Exercise for the Reader(tm) to figure out the best path to avoid memory unsafety given races in line with other Go priorities (as a starting point, imagine a ton of expensive atomics accessing certain pairs on the heap, downgraded to regular loads/stores if the compiler can prove it's safe). But I was trying to say it seems harder to get there now that certain things are set in stone and regressing perf of anything is going to be bad for some user out there.
You'd probably hit other walls. I'm def. not trying to say it's straightforward!
Shared state:
ArrayList A with intial buffer B and length L
Thread 1:
Grows A and replaces B with buffer C, length with M
Thread 2:
(Earlier loaded B into a register)
Code says write to A[M-1]. JVM checks M-1 against M but uses B as it was already stored in a register. Now you have a buffer overflow.
Except, people on this thread seem to think this is impossible. How does Java prevent this?
In most managed languages to 'properly' deal with it you typically need to accept locking on some level (like with ConcurrentDictionary in C# or ConcurrentHashMap in Java).
One semi-elegant (yet internally complex) solution I've seen; in C# there is the language-ext library. All the collection types are structs (value types where things like .Add() return a new struct) however the struct itself just holds an implementation to a class. This works well from a user standpoint but the internals of the code is... interesting to say the least and can still lead to a lot of allocations if you're not careful.
Java is like Go in that regard. That's why you have to use ConcurrentHashMap instead of HashMap for instance.
https://stackoverflow.com/questions/51102644/what-the-worst-...
Java may be non-deterministic, or exhibit unintended behaviour in response to a race condition. UB is strictly worse because it could do either of those things... or indeed anything else, segfaulting being the least of your concerns.
> Undefined behavior (UB) is the result of executing a program whose behavior is prescribed to be unpredictable, in the language specification to which the computer code adheres.
Can we predict the behavior of a Java program with a data race?
That post from Russ Cox (again) explains quite well why Go is not memory safe in presence of data races, and what should be changed to fix this: https://research.swtch.com/gorace
Here's a better definition: https://en.cppreference.com/w/cpp/language/ub
A few years back someone at Dropbox mentioned tricky prod-only races were a real problem for them sometimes, for instance: https://about.sourcegraph.com/go/go-reliability-and-durabili...
That's where I start dreaming about best-effort detection of races in production binaries or even just reducing the chances of torn reads. Years back there might have been other options, like explicit private vs. shared heaps with more controls on the shared heap, but there seems to be a more restricted set of choices now.
https://blog.stalkr.net/2015/04/golang-data-races-to-break-m...
> We have to admit that exploiting this requires a fairly specific situation in which there is a data race we could trigger and some structs with function pointers around.
I would note that Java is designed to run untrusted code, so fat pointers would be unacceptable as the attacker could easily craft the code required to trigger the race. Go does not claim to provide sandboxing of untrusted programs.
> Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs.
Source: https://docs.oracle.com/javase/8/docs/api/java/util/Vector.h...
This is conceptually similar to the race detector in Go, which is also a debugging tool, but without strict guarantees.
Note that this isn't actually a real exception. maps are implemented in the go stdlib using "unsafe", in addition to compiler magic for the generics part.
I believe that it holds that if you don't use "unsafe" then your program is memory and type safe, it just happens that the go stdlib and runtime use "unsafe", so you can't safely use them either.
I get that this is a distinction without a difference.
It's still the job of the code author to make it so you can safely use that code, despite internal use of unsafe.
data-races that can't cause undefined behavior are a different class, closer to what java has, and iiuc that's what you get without unsafe.
I don't think the distinction matters, but I think it exists.
https://blog.stalkr.net/2015/04/golang-data-races-to-break-m...
[1] unless there is an additional indirection, I don't really know much about go.
That second sentence is one of the painful things about the Go memory model--Go is type and memory safe if you avoid explicitly unsafe things, except for data races on those structures.
I wonder what you can practically do to mitigate that at this point; doesn't seem like you go back and change things to be like Java. Can there be some best-effort detection for races in those specific places that's fast enough to run in prod (like I think they did for maps)? I also recall some Go mailing-list post I can't find suggesting that an SSE 128-bit move, while not guaranteed not to tear, seemed to tear only rarely in their tests on the chips of the time; is that true and is there anything there? I imagine fully preventing the problem with atomic reads/writes of these pairs from the heap is a clear no-go performance-wise, and might involve ABI-changing alignment guarantees.
[Edit: SSE2 behavior is discussed here https://stackoverflow.com/questions/7646018/sse-instructions... which is mentioned in a comment at https://research.swtch.com/gorace which seemed to inspire it. Chance of a race seemed to depend on CPU model at the time, bad on the old Core Duo, better on some server chips another answerer tried. Good on my laptop FWIW!]
Probably hard for anything to happen here because any performance regression is hard to swallow, the race detector is close enough for many, and anything you do won't be a 100% solution anyway. Which is a bummer.