News for hackers. Stuff that matters.
Re:Are Go maps sensitive to data races? by grabcocque on Monday December 07, 2015 @06:26AM (#10688763)
"no, there is nothing wrong with Go’s map implementation."

"Getting your locking wrong will corrupt the internal structure of the map."

Ahem.

If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong.

Re:Are Go maps sensitive to data races? by iand on Monday December 07, 2015 @06:28AM (#10688770)
The two sentences are consistent once you read that Go maps are not designed to be safe for concurrent access.
Re:Are Go maps sensitive to data races? by grabcocque on Monday December 07, 2015 @06:33AM (#10688783)
Java's (pre-concurrent) HashMaps aren't designed for concurrent access.

Do you know what they do? They throw a ConcurrentModificationException.

Do you know what they don't do? Silently corrupt themselves.

Re:Are Go maps sensitive to data races? by vbezhenar on Monday December 07, 2015 @06:38AM (#10688796)
From HashMap JavaDoc: Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs.

So even while Java tries to find concurrency bugs, it does not promise it.

Re:Are Go maps sensitive to data races? by lmm on Monday December 07, 2015 @08:01AM (#10689135)
Indeed it doesn't promise it. But in practice it's very good at it - the reason they have to put that note in there is because if you made a program whose correctness depended on it, it would be a long time before you saw the bug.
Re:Are Go maps sensitive to data races? by 4ad on Monday December 07, 2015 @06:39AM (#10688799)
Not to mention that Go does have the race detector, which does the same thing in this case (although it does so much more in general).
Re:Are Go maps sensitive to data races? by alkonaut on Monday December 07, 2015 @06:40AM (#10688801)
That check probably comes at a performance cost that the Go standard library designers found unacceptable. You don't even need to be modifying it concurrently to read inconsistent state, it's enough to write on one thread and read on another, while the map is updating internally. So in order to prevent inconsistent reads you would have to check for concurrent access on all reads. The .NET BCL Dictionary is just like the Go one: it does no checking for you and will silently corrupt or return an inconsistent view if you access it concurrently.

The whole reason there are two implementations of the dictionary (Concurrent and Non concurrent) is that the performance cost of the concurrent implementation is too high to carry for the non-concurrent one. Depending on the cost of checking for concurrent access, a lot of the gain of using a simpler non-concurrent one could be lost.

Re:Are Go maps sensitive to data races? by binarycrusader on Monday December 07, 2015 @05:46PM (#10693377)
Well, they believed it was worth adding something (from a commit today):

https://github.com/golang/go/commit/50c5042047be3af36e7bb478...

Re:Are Go maps sensitive to data races? by alkonaut on Tuesday December 08, 2015 @08:30AM (#10696189)
That's interesting. Would have liked seeing something like that in the Java and .NET equivalents.
Re:Are Go maps sensitive to data races? by shellac on Monday December 07, 2015 @06:57AM (#10688861)
The check is actually very cheap -- you can check the source yourself -- but as you might expect it won't catch all unsafe use.
Re:Are Go maps sensitive to data races? by alkonaut on Monday December 07, 2015 @07:07AM (#10688890)
Like you suggest you can make a cheap check that sometimes triggers an exception. Or an expensive check that always triggers on all concurrent access. As far as I can see in the java HashMap source, the concurrent modification check is actually only done for the enumerator (?) so the problem of multiple concurrent insertions, or even reading-while-inserting will never be caught anyway. Just like Go, or .NET.

A check for single thread access (which is even stricter than non-concurrent access which allows several threads as long as they aren't used concurrently) would be pretty cheap: store the thread ID on creation and then verify on reads and writes.

Failing on concurrent access otherwise on all reads and writes would basically mean that you add a lock to the write operation, and fail any reads while anyone is writing. This however is close enough to making it a full concurrent map, so it isn't worth it.

Re:Are Go maps sensitive to data races? by noselasd on Monday December 07, 2015 @07:42AM (#10689059)
A Java Hastable is threadsafe, and is documented to be so

A Java HashMap is not threadsafe and is documented to be so, and can silently corrupt data, can go into an infinite loop etc. if you mess with it from several threads.

Both leaves you with a lot of potential for races in application logic though.

Re:Are Go maps sensitive to data races? by coldtea on Monday December 07, 2015 @06:37AM (#10688792)
Yeah, but the parent's phrase was:

"If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong".

If we accept his premise, then the fact that "they wasn't designed for that" is no excuse. It's like selling a kids toy that has tiny choke-inducing parts. In that case, just saying: "It wasn't designed to be eaten by kids" is not really an excuse.

Re:Are Go maps sensitive to data races? by alkonaut on Monday December 07, 2015 @06:49AM (#10688830)
A lot of comments that just seem to ignore the fact that noone would actually want standard collections to be thread safe! Concurrent collections are slow. That's why class libraries use different types for a therad safe collection and a regular non-thread safe collection. If you are using concurrency you have to use concurrent collections.

You could argue that if most Go code is concurrent then the regular collections should be the thread safe ones, and for code that is known to be single threaded there would be simpler/faster non-concurrent collections. That is a completely valid argument to make, but I think that design would be too confusing for new Go developers, since in most other languages the standard collections are the non-concurrent ones, and the concurrent ones are special, and it's an explicit design goal of Go to be easy to adopt coming from another language.

Re:Are Go maps sensitive to data races? by grabcocque on Monday December 07, 2015 @07:33AM (#10689005)
They're really not though. Java has ConcurrentHashMap and isn't slow. Clojure has its persistent HashMap and it isn't slow. Scala has, TWO (?) concurrent hash maps in its core and neither is slow.

You know, two thirds of the way through the second decade of the 21st century, it's okay to spend a few cycles not corrupting your data structures.

Re:Are Go maps sensitive to data races? by alkonaut on Monday December 07, 2015 @07:41AM (#10689045)
There are both memory and performance considerations. For a server/db web application this may not be a problem, but for e.g. some logic in a game engine that runs every frame on a mobile device, then using 16 extra locks per map for example isn't a good idea. Nor is having 5% performance overhead.

I can agree with an argument that thread-safe should be default and specialized/fast collections should be optional though.

Re:Are Go maps sensitive to data races? by astrafoo on Monday December 07, 2015 @12:25PM (#10690889)
People who care about 5% performance overhead will never use Go in the first place.
Re:Are Go maps sensitive to data races? by alkonaut on Monday December 07, 2015 @02:05PM (#10691783)
That's just not true. Regardless of language there is always a portion of the code where you need to be careful with performance. I have spent endless hours tweaking map performance in .NET, for example.
Re:Are Go maps sensitive to data races? by 4ad on Monday December 07, 2015 @06:47AM (#10688827)
First, that statement was not present in grandparent's post when the parent posted his message.

Second, that premise is completely untrue. Go, and most other languages give you more-or-less orthogonal primitives and let you combine them. It's okay for your types not to be thread safe, if by combining them with some other primitive you can make them thread safe.

Every other language does this. Some offer you a concurrent map along with a non-concurrent map. For many reasons I won't get into, Go only has one kind of map and lets the user do the rest (which is an extremely easy idiom in Go). Note that even though many languages conveniently offer concurrent maps, no languages with mutable state that I know of offer concurrent integer and concurrent structs. The user is still responsible for ensuring the safety of those.

This is perfectly fine because the grandparent's premise is wrong. In concurrent language, by far the most common case is by data to be owned by a goroutine/thread/whatever. It's very easy to reason about code this way, and it's the way you are encouraged to write code.

Only when you need to share data you need to worry about concurrency-safety, and in that scenario you have available all the tools to ensure it.

Re:Are Go maps sensitive to data races? by lambda_cube on Monday December 07, 2015 @07:25AM (#10688974)
> no languages with mutable state that I know of offer concurrent integer and concurrent structs

Java does for integer http://docs.oracle.com/javase/8/docs/api/java/util/concurren...

You can find similar classes in the java.util.concurrent.atomic package. http://docs.oracle.com/javase/8/docs/api/java/util/concurren...

I'm not sure what requirements you have for a concurrent struct, but Java has classes to atomically manipulate int and long fields of a class.

I'm not arguing your general point (in fact, I agree with it), I'm just supplying some extra information of a language that you apparently don't know.

Re:Are Go maps sensitive to data races? by arethuza on Monday December 07, 2015 @06:45AM (#10688822)
I think there is a big difference between "being good for concurrency" and preventing all concurrent programming problems - concurrency is hard and while go does seem to provide a lot of nice structures to make it more straightforward it can't, and doesn't appear to claim to, hide all the complexities of concurrency from developers.
Re:Are Go maps sensitive to data races? by goykasi on Monday December 07, 2015 @07:14AM (#10688919)
I see nothing wrong with how Go's map implementation was described. Go has never claimed to be completely safe for concurrent operations. Deadlocks, inconsistent state, etc are more than easy to accomplish. But, it still stands up to a "good for concurrency" claim by offering nice primitives for writing concurrent code, a race detector and explicitly stating in the stdlib documentation which libraries are thread-safe -- libraries are to be considered not safe unless explicitly said to be.
Re:Are Go maps sensitive to data races? by JulianMorrison on Monday December 07, 2015 @07:00AM (#10688876)
Go's idiomatic way to handle access to maps is: at any one time exactly one thread owns the map. Either pass it around, or have a gatekeeper thread.
Re:Are Go maps sensitive to data races? by 4ad on Monday December 07, 2015 @06:32AM (#10688782)
> Ahem

Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++.

The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios.

But I guess it's fashionable to snark on message boards than rather understand all these facts.

Re:Are Go maps sensitive to data races? by titzer on Monday December 07, 2015 @06:52AM (#10688845)
Actually Java guarantees atomic accesses for integer-sized fields and references at the language level. They aren't race-free, but they are atomic. And really, atomicity is the main property that the OP had in mind when posting. Since you mentioned C++, it also does guarantee certain-sized certain-alignment loads and stores are atomic.

In Java there is also the plethora of thread safe data structures available in java.util.concurrent. They are written by experts and are pretty darn fast (TM). It might have been reasonable to use one of these as the basis for Go map implementations, but the language designers chose to assume unsynchronized access to shared maps is a design bug in applications.

Re:Are Go maps sensitive to data races? by jeremyjh on Monday December 07, 2015 @07:02AM (#10688881)
Java chooses safety over performance. It is not free to check for concurrent access in a non-concurrent structure. In some languages, we let the programmer take on the choice of which costs to bear. There is not very much that I like about Go, but there is nothing wrong with having non-concurrent data structures that are documented as such and fail in non-defined ways when there is shared data access.
Re:Are Go maps sensitive to data races? by azth on Saturday December 12, 2015 @12:58AM (#10721964)
> Java chooses safety over performance.

Yet manages to be faster than golang.

Also, the two need not be mutually exclusive as you're implying.

Re:Are Go maps sensitive to data races? by viraptor on Monday December 07, 2015 @06:59AM (#10688870)
C got atomic values in C11: http://en.cppreference.com/w/c/language/atomic

Not that I disagree with your comment in general, but atomic values do exist in many languages.

Re:Are Go maps sensitive to data races? by AYBABTME on Monday December 07, 2015 @07:05AM (#10688888)
Re:Are Go maps sensitive to data races? by 4ad on Monday December 07, 2015 @07:37AM (#10689025)
Thanks for the reference, I knew about C11 atomic types, and, I should have use a better phrasing. Note that C11 atomics are still not applicable to multi-word aggregate types, like structs. You'd use atomics in C11 (in the type system), for what you'd use sync/atomic (like other poster mentioned) in Go (outside the type system).
Re:Are Go maps sensitive to data races? by seanmcdirmid on Monday December 07, 2015 @07:18AM (#10688940)
32 bit data types are read and written atomically on the CLR. Pointers are read/written atomicly on most ANY platform, or you couldn't build any kind of higher level concurrency abstraction. If not, the language could never provide memory safety.
Re:Are Go maps sensitive to data races? by 4ad on Monday December 07, 2015 @07:32AM (#10689003)
You need a way to do atomic reads and writes, but you don't need to have atomic types. In Go you just use sync/atomic which implements atomicity outside the type system.

When you use "regular" operations, you don't get atomicity, but what you get is well-defined, and explained here: https://golang.org/ref/mem.

Re:Are Go maps sensitive to data races? by grabcocque on Monday December 07, 2015 @06:35AM (#10688787)
I apologise for expecting better of language designers. After all, the solutions to these sorts of issues has only been around for a couple of decades. Far too soon.
Re:Are Go maps sensitive to data races? by alkonaut on Monday December 07, 2015 @06:43AM (#10688813)
This is a library thing, not a language thing. And it's a performance tradeoff: they could have shipped with a concurrent map only, that is 1/10 as fast as the current one, but never causes a data race. However everyone that isn't doing anything concurrently would be paying for it. Not very clever. I trust the class library designers thought this through. They reached the same conclusion as pretty much every other library designer in every other language. For the same reasons.
Re:Are Go maps sensitive to data races? by titzer on Monday December 07, 2015 @06:53AM (#10688846)
Data races are not the same thing as non-atomic accesses.
Re:Are Go maps sensitive to data races? by alkonaut on Monday December 07, 2015 @07:01AM (#10688877)
Sorry, I interpreted the statement as being about concurrency in maps, not about atomicity.
Re:Are Go maps sensitive to data races? by netheril96 on Monday December 07, 2015 @06:50AM (#10688835)
You are not expecting better. You are expecting worse. Plenty of use cases involve maps without concurrent modification, e.g., when used as a local variable. Your "better" version of maps will slow down all of them.
Re:Are Go maps sensitive to data races? by jrockway on Monday December 07, 2015 @06:40AM (#10688803)
All the language designer can do is throw in a lock that you may or may not need if you're already locking properly. ConcurrentModificationException is nice, but comes at a performance cost.
Re:Are Go maps sensitive to data races? by lazyjones on Monday December 07, 2015 @07:17AM (#10688936)
The explanation for this design decision is reasonable IMHO: https://golang.org/doc/faq#atomic_maps