Over & over I see language aficionados ding Golang for not taking advantage of immutability and for allowing shared data --- or, in your case, going a step further and reducing all communication among processes in Golang to instances of synchronized sharing.
What I'd like to know is: why don't all those hundreds of thousands of lines of concurrent Golang code out there, including all the library code I can just "go get" and whose authors have been encouraged by Rob Pike to use, basically, threads with near total abandon (watch his video about designing a lexer!) --- why don't all those libraries and programs randomly deadlock and corrupt themselves all the time?
Because my experience is that Golang code is quite a bit more reliable than, for instance, Python code.
What am I missing? The "share by communicating" model in Golang seems to work pretty darn well, especially given the extent to which Golang begs programmers to make programs concurrent.
† OK probably you more than me but still.
The simplest answer would be "they do." In aphyr's recent presentation on Jepsen, where he tested etcd (a Go database implemented on top of Raft), he noted that when he started using it he encountered a ton of easily reproducible races and deadlocks (which he sarcastically noted was surprising because he thought goroutines were supposed to make concurrency issues a thing of the past).
I am not saying that Go channels don't help the situation at all--and the inclusion of a race detector doesn't hurt either--but you still have plenty of ways to shoot yourself in the foot. The thing that probably helps most is that GOMAXPROCS is 1 by default, since data races are a multicore phenomenon in Go.
Maybe I'm wrong and etcd was riddled with concurrency problems between the goroutines of a single etcd process?
In any case: as anyone who has worked on a large-scale threaded C++ codebase can tell you: Golang programs simply do not exhibit the concurrency failures that conventional threaded programming environments do. It would be one thing if Golang code only used concurrency for, say, network calls. But goroutine calls are littered throughout the standard library, and throughout everyone's library code.
But there are better things out there -- isolated heaps (Erlang), borrow checkers (Rust), stronger type systems and immutability (Haskell) etc. There are no magic unicorns so those things often come at a price -- sequential code slowdown.
Getting back to go. One can of course say, "Oh, send only messages. We are all adults here. Let's just agree to be nice. Stop sharing mutable memory between goroutines!" But all it takes is "that guy" or "that library", doing it "that one time" and then there are crashes during a customer demo or during some critical mission. It crashes and then good luck trying to reproduce it. Setting watchpoints in gdb (or the equivalent Go tool), asking customers "Can you tell me exactly what you did that day. Think harder!" and so on.
Also, as others have pointed, with Golang though, often it is run with just one OS thread backing all the concurrency. So many potential races could be just be hidden.
There is also some confirmation bias involved. When something is broken, often authors don't write blogs about it, don't advertise. They fix it, and move on. So maybe a lot of programs are full of concurrency bugs but just nobody is blogging about it. They've invested time and energy into learning a new ecosystem and now they have to blog about its flaws and so on. That is hard to do.
Another observation is that when spending a lot of time debugging and handling segfaults, pointer errors, user-after free errors, concurrency issues, that becomes the default and expected view of how programming works. It becomes hard to imagine how it could work another way. It becomes obvious that weeks would be spent tracking one concurrency bug or having to add cron jobs to watch for crashed programs and restart them because the system is so complex and non-deterministic, replicating the bug is too hard.
It doesn't, it was just grouped with the other two. But immutiblity-by-default could lead to slowdown.
Furthermore, since the compiler's knowledge of mutability is directly related to its knowledge of ownership, one could argue that immutability actually makes code faster by dint of providing greater aliasing information (e.g. `restrict` in C) to the optimizer (though the Rust compiler has yet to actually leverage this optimization).
func (*Mutex) Lock
Lock locks m. If the lock is already in use,
the calling goroutine blocks until the mutex is
available.
http://golang.org/pkg/sync/It seems to me this is a valid alternative to [rigidly] sticking to pure message passing.
EDIT: The mutexes themselves are easy enough to get right, it's the systems using mutexes that are too hard to get right.
Or the other way around, what mutex protects a piece of data (or even that a piece of data should be protected at all), so it's easy to forget it and just manipulate a bit of data without correctly locking it.
I was pleasantly surprised to discover that Rust's sync::Mutex owns the data it protects, so you can only access the data through the mutex (and the relation thus becomes obvious).
Well aside from the deadlock or priority inversion of sorts that should work. Just like it would work in C/C++/Java etc.
The real problem is when the shared data is not controlled by a mutex, but should be.
My point wasn't that Go is _worse_ than contemporary languages like C++ and Java when it comes to data races, only that it doesn't eliminate them. Which, again, it doesn't claim to. Rust does, and it is an important difference between the two languages. Because data race freedom with cheap mutable state requires a garbage-collection free subset of your language [1], I think it's unlikely that Go will ever guarantee this.
[1] as noted by Niko Matsakis at http://smallcultfollowing.com/babysteps/blog/2013/06/11/on-t...
You mean file a bug like "issue #1935 -- stop sharing memory between goroutines" (I just made it up to be silly there is no such bug report)
In other words, they have explicitly designed in the ability to share memory between goroutines. You can certainly file an issue or bug report about, somehow I doubt that will lead to much but being kickout and laughed at.
One can also just have 2 goroutines wait on each for results and that's a deadlock. Maybe there is a tool to detect that would be nice. Do you know of one?
https://code.google.com/p/go/source/browse/src/runtime/proc....
I'm hoping that as Swift evolves over time, it will slowly become that sort of language. Right now it's pretty hard to write any real-world code in Swift that doesn't work with the Cocoa frameworks, and the Cocoa frameworks are typed in an objective-c-compatible way (even if new frameworks are written in Swift they'll need to maintain obj-c compatibility), which means you don't get the strong typing that's necessary for this property. Pure Swift code has the potential to behave like this, although you probably need to avoid the ImplicitlyUnwrappedOptional feature (the ! suffix on types), which of course primarily exists for ease of obj-c integration anyway.
I'm bringing up Swift because, with Apple's backing, it's very quickly becoming a "mainstream" language. I put that in quotes because it is only usable with iOS and OS X programming (for now at least), but iOS is large enough that obj-c should be considered a mainstream language despite the fact that almost nobody outside of iOS/OS X uses it, and therefore as Swift supplants obj-c it becomes appropriate to call it mainstream.
Regarding parallelism, I've been in love with Rust for a long time now, and one of the biggest reasons is because Rust makes parallelism safe. As an iOS/OS X programmer by trade, I think thread safety is far and away the biggest elephant in the room. Despite the fact that we've known that multithreading is the future for years, and despite the wonderful Grand Central Dispatch library on iOS/OS X, most programmers still think in a single-threaded mindset and don't even consider how their code should operate if invoked on a separate thread. This was one of my bugaboos with Go back when I was using that language (which was from the day it was announced right up until I discovered Rust, though admittedly my usage was in hobby projects and nothing serious).
I applaud the fact that Go has a data race detector now, which I used to finally uncover a lurking data race that plagued one of my programs for months (and which was ultimately caused by the Go library using two goroutines where I expected one, and therefore data which I expected to be on a single goroutine was actually mutated from two goroutines simultaneously). But I think Rust is absolute proof that a modern language can be designed such that data races are prohibited at compile-time without sacrificing any language flexibility.
I don't think it is at all obvious that Haskell won't become mainstream. It's already exerted a tremendous influence over many other mainstream languages and there's only so long that can happen before people just start going directly to the source of the innovations (or one of its direct descendants).
To wit, Lisp never became mainstream despite exerting a huge influence. Likewise Smalltalk.
I also said "or one of its direct descendants" (like Agda or Idris in all likeliness).
> To wit, Lisp never became mainstream
Clojure doesn't count? And the good parts of Perl, Ruby & Javascript are essentially Lisp without the homoiconicity.
Perfect example. Clojure != Lisp. But it is a pretty obvious descendant.
Beyond that, I'm not quite sure Clojure counts as "mainstream" yet. According to the TIOBE Index, it doesn't even rank in the top 50 languages. Heck, the top 20 includes R, and Dart, neither of which I would call "mainstream" (I'm actually really surprised at how high R is ranking). I don't know how significant that is, though the TIOBE Index is measuring "number of skilled engineers world-wide, courses, and third party vendors" and that seems like a reasonable approximation for "mainstream" to me.
Clojure is not married to the JVM either-- in fact, it has been hinted that it would jump ship if something better comes along or the current situation becomes less viable. Furthermore we already have a dialect of Clojure called ClojureScript which targets JavaScript/node/V8.
And, I look at the JVM as really merely a library/API/runtime. C++ has STL and stdio and such and they are not part of the language proper but rather merely libraries for interacting with the underlying operating system (in a platform independent way). The same is true for the JVM with respect to Clojure and Scala et al.
Java happened. Business were already in the process of adopting Smalltalk.
Hotspot is a Smalltalk JIT compiler reborn.
Eclipse is Visual Age for Smalltalk reborn. It still keeps the old Smalltalk code browser.
Yeah, but nothing in it is Smalltalk-specific. It's not like Smalltalk survives in the mainstream because of Hotspot (in the way that, say, Algol survives).
[edit: survives, not "survices"]
It's not like SUN was the only player in town. IBM pushed Smalltalk IIRC.
I think this (from StackOverflow) tells a more comprehensive story):
• when Smalltalk was introduced, it was too far ahead of its time in terms of what kind of hardware it really needed
• In 1995, when Java was released to great fanfare, one of the primary Smalltalk vendors (ParcPlace) was busy merging with another (Digitalk), and that merger ended up being more of a knife fight
• By 2000, when Cincom acquired VisualWorks (ObjectStudio was already a Cincom product), Smalltalk had faded from the "hip language" scene
http://stackoverflow.com/questions/711140/why-isnt-smalltalk...
Eclipse 1.0 was Visual Age for Smalltalk redone in Java.
If Java hadn't appeared in the scene, maybe even with those cat fights, the language would have become mainstream anyway.
This just speculation from my part.
[1] The famous "private protected" that was accepted in the very first release.
Which is quite different.
Just go ahead and learn the typeclass hierarchy and such-- it really is quite a useful higher level of abstraction in whatever language you choose. And it definitely will enter the mainstream (even more than is already has [Swift, Scala & C# all have monadic constructs]).
> Haskell's memory footprint is extremely hard to reason about.
And you'd probably want to also throw runtime in there as well.
I think this is relative-- it's not "extremely hard" for everyone. Also, many structured programmers found object orientation "extremely hard" but somehow the industry managed to progress through that era.
Haskell is very good at a), and not bad at all at b). With the help of the profiler it shouldn't be that hard to determine a program's bottlenecks/leaks and fix them, as with any other language.
BTW, since you mention http, I have this reading on my back burner [1] but from skimming it found that for certain payloads a haskell http server may perform better than nginx (1.4, circa 2013?), which is an impressive feat.
I'd have thought that someone would pick the higher level language first, and go to the lower lever when in need of more control.
If it is obvious then you should be able to explain why in a few sentences.
One sentence. Is that sufficient?
The basic rule is if you see a segfault or a data race, search your code for any `unsafe {}` blocks, and the cause will always be found inside one of those. And as a corollary, never use `unsafe {}` if you can possibly avoid it.
Just to add a clarification on the implied comparison: Rust only protects against data races at compile time. Other forms of race conditions are still possible.
The way structs are supported, it feels much more like objects than C structs. I prefer it much more than say, OOP classes. It's something you need to immerse yourself in to appreciate, IMO.
What about "user created objects" would you miss?
Go isn't bad for writing routine server-side web stuff that has to scale and run fast, which is why Google created it. Go is a modern language with a dated feel. No user-defined objects, just structs. No generics or templates. It was designed by old C programmers, and it looks it. Go has generic objects - maps and channels - and syntax for creating object instances - "Make". Only the built-in generics are available, though; you can't write new ones.
Go's "reflection" package thus tends to be overused to work around the lack of generic. This means doing work for each data item at run time for things that could have been done once at compile time. "interface{}" (Go's answer to type Any from Visual Basic) tends to be over-used.
Go (especially "Effective Go") has a lot of hand-waving about parallelism. Go's mantra is "share by communicating, not by sharing", but all the examples have data shared between threads. The channels are just used as a locking mechanism. Race conditions are possible in Go, and there's an exploit which uses this. (That's why Google AppEngine limits Go programs to single threads.) Go doesn't use immutability much, which is a lack in a shared-data parallel language with garbage collection. If you can make data immutable, you can safely share it, which is a way to avoid copying without introducing race conditions.
Rust, like Erlang, takes a much harder line on enforcing separation and locking. I haven't used Rust myself yet, so I can't say more on what it's like to use it. My hope is that Rust will provide a solution to buffer overflows in production code. After 35 years of C and its discontents, it's time to move on. I really hope the Rust crowd doesn't fuck up.