News for hackers. Stuff that matters.
Re:Channels in Golang by dispose13432 on Saturday December 24, 2016 @10:58PM (#13252828)
I know it's a controversial point among users, but I think that if Generics were such a show stopper, Google would have implemented them.

Go was invented for internal Google use, and then shared with the world, and Google is an engineering company. If engineers would have rebelled, higher management probably would have done something about it.

-----

Although I don't understand why they keep fighting it. A simple style generics (Java style) shouldn't make code that ugly.

Re:Channels in Golang by scottlamb on Sunday December 25, 2016 @12:46AM (#13253130)
> Go was invented for internal Google use, and then shared with the world, and Google is an engineering company. If engineers would have rebelled, higher management probably would have done something about it.

Not necessarily. Go was invented for internal Google use, but most of Google's code is still in C++ and Java. New projects are often started in C++ and Java, even, as they depend on libraries written in those languages. I think this would be true even if Go were perfect; switching languages is hard. But it means you shouldn't assume Google has solved whatever problems one would encounter while implementing Google product X in Go. Or that all Google engineers believe Go is a good language; as with any large group of programmers, there are people with opposing viewpoints about the virtues of various programming languages.

Google management doesn't tend to impose top-down technical decisions such "product team X must use Go" [1] or "the Go team must add language feature X". They generally give broad direction and then trust the engineers doing the work to find the best technical approach to accomplish the task.

[1] Although you generally must stay within the handful of approved languages: C++, Java, Go, JavaScript (for frontend stuff), Python (mostly for internal stuff, and getting less socially acceptable over time). One reason for this list is that when you write a Google system in a new language, you need a bunch of support from the rest of the company to do it: build infrastructure, libraries, etc.

Re:Channels in Golang by twic on Sunday December 25, 2016 @09:30AM (#13254184)
> Go was invented for internal Google use

Go was invented by a small group at Google with the aim of solving some of Google's problems as they saw them. It's not as if the whole of Google came together and collectively designed the language they needed. I wouldn't read too much into the fact that it happened to be invented at Google, rather than wherever else Rob Pike happened to be working at the time.

Re:Channels in Golang by catnaroek on Saturday December 24, 2016 @11:27PM (#13252922)
Parametric polymorphism alone is a simple yet very powerful feature, but this assessment changes drastically as soon as subtyping (which both Go and Java have) enters the picture. The combination of parametric polymorphism and subtyping creates the pesky issue of determining whether Foo<T> should be a subtype of Bar<U>.

Java-style generics are a lot of things (notably “bolted on”), but “simple” certainly isn't one of them. Wildcards are complex. F-bounded polymorphism is complex. Non-denotable types are complex.

Re:Channels in Golang by pcwalton on Sunday December 25, 2016 @12:20AM (#13253066)
Go doesn't have subtyping. It has coercions, which are not the same thing, and they do not lead to issues around variance.

Variance is straightforward anyway. A lot of things in Go, like the semantics of "nil" and zero values, are more complex than variance. And, as a designer, if you're worried about the complexity, just make all type parameters invariant, like C++ does. It's a bad excuse to not have generics at all.

Rust works exactly the same way as Go (coercions, no subtyping), except that there is subtyping in Rust via the region system. But Go doesn't have that. No region system, no subtyping.

Re:Channels in Golang by jzelinskie on Sunday December 25, 2016 @01:51AM (#13253277)
I googled for a bit and didn't find anything current about Rust's region system from an implementation perspective. Is there a good place to read more in depth about design decisions, but containing less process than the RFCs and PRs?
Re:Channels in Golang by eddyb on Sunday December 25, 2016 @02:07AM (#13253317)
The only document I can think of outside of Niko Matsakis' blog (http://smallcultfollowing.com/babysteps/) would be https://github.com/rust-lang/rust/blob/master/src/librustc/i...
Re:Channels in Golang by stjepang on Sunday December 25, 2016 @11:49AM (#13254635)
There is a chapter in Rustonomicon on subtyping and variance: https://doc.rust-lang.org/nomicon/subtyping.html
Re:Channels in Golang by catnaroek on Sunday December 25, 2016 @01:05AM (#13253172)
What distinction between coercion and subtyping are you making? Aren't Go structs subtypes of every interface they conform to?

And I don't think making all type parameters invariant eliminates any complexity. It just pushes it onto the user.

Re:Channels in Golang by burntsushi on Sunday December 25, 2016 @07:14AM (#13253866)
It would be the other way around. It is intuitive to say that interface{} is a subtype of all other types, but it is easy to show a contradiction. Assume that interface{} is a subtype of int, which means interface{} can be used in place of int in any type. So define a function with type func(xs []int). If you pass a []interface{} to this function you will get a type error. More concretely, []int and []interface{} have different in memory representations.
Re:Channels in Golang by computerex on Sunday December 25, 2016 @07:56AM (#13253970)
Can you explain why the empty interface would be a subtype of [] int and not the other way around? Intuitively, I think it's the other way around, as every type implements interface{}.
Re:Channels in Golang by burntsushi on Sunday December 25, 2016 @09:08AM (#13254116)
I guess neither direction is especially intuitive because structural typing makes the relationship a bit strange. In both cases, it's easy to show a contradiction. `interface{}` can't substitute `int` in `[]int` and `int` can't substitute `interface{}` in `[]interface{}`.
Re:Channels in Golang by catnaroek on Sunday December 25, 2016 @09:03PM (#13256497)
I'm not sure how you concluded that I said interface{} is a subtype of all other types. Perhaps you meant supertype?

Also, from “Foo <: Bar”, you can't automatically conclude that “[]Foo <: []Bar”. That requires the additional assumption that [] is a covariant type constructor, which in Go it isn't.

Re:Channels in Golang by burntsushi on Sunday December 25, 2016 @11:27PM (#13256920)
I didn't conclude that at all. It is trivial to produce a contradiction if you assume A <: B because Go will not let you substitute A in place of B for every type in which B occurs. Therefore A is not a subtype of B. This has nothing to do with []A <: []B.
Re:Channels in Golang by catnaroek on Sunday December 25, 2016 @11:33PM (#13256934)
But that's not at all what subtyping means! It doesn't mean that you can substitute every occurence of “A” with “B” in the program text. It means that every term of type “A” also has type “B”. (Source: I have TaPL p. 182 right in front of me.)
Re:Channels in Golang by burntsushi on Monday December 26, 2016 @11:34AM (#13258757)
OK, I see where I was wrong now. You are right to emphasize the word "term" because that's precisely the detail I was missing. I can no longer come up with a contradiction.

The next obvious step is getting an answer to "what is the meaning of saying Go doesn't have subtyping?", which is essentially what you asked above:

> What distinction between coercion and subtyping are you making?

But this is a question for pcwalton, not you. :-) However, I have some thoughts on the matter.

While exploring the extent of my wrongness, I found it very interesting that the Go language spec doesn't mention the words "subtype" or "polymorphism" or "structural" at all. The FAQ mentions "subtype" and "covariance," but doesn't lead to anything conclusive. What I think this means is that the language designers of Go probably don't think of interfaces as being tied to the formal notion of subtype, but rather that of assignability or coercions. This just takes us in a circle though, which leads me to guess that this is a simple matter of two groups of people having different names for the same thing.

Re:Channels in Golang by catnaroek on Monday December 26, 2016 @12:32PM (#13258984)
The litmus test for “typeness” is to ask whether it makes sense to ask whether a term has the type in question. For example, it makes sense to ask whether 2 has type string, and the answer is “no”. But it doesn't make sense to ask whether 2 has type 3 - the closest thing to an answer is “what the hell I don't even?” Then:

(0) Structs and interfaces are unarguably types.

(1) If a struct S has all the methods required by an interface I, then every term of type S also has type I.

As for coercions, they are one possible implementation of subtyping. Another possible implementation is guaranteeing that, whenever “Foo <: Bar”, then “Foo” and “Bar” have a common in-memory representation.

But languages shall never be conflated with their implementation details!

Re:Channels in Golang by burntsushi on Monday December 26, 2016 @12:50PM (#13259052)
Yes, I think pcwalton saying that Go doesn't have subtyping is very confusing. What I think he was trying to say was that Go's subtyping is strange because everything is invariant, and that the use of coercions makes the questions you initially raised perhaps not so interesting.

For example, if you start with some base assumptions about which type constructors in Go are invariant, e.g., pointers, slices, arrays, etc., then you could feasibly come up with a rule like this: If T <: U, then Foo<T> <: Foo<U> if Foo's type parameter doesn't appear within an invariant type constructor in the definition of Foo. If a rule like that is all you need (and those types of rules have precedence in Go, like whether a type permits equality comparisons or not), then I think that more or less backs up pcwalton's point that subtyping isn't necessarily the thing you have to worry about when adding parametric polymorphism to Go. It's the other stuff, like the mentioned zero values and presence of `nil`.

Re:Channels in Golang by catnaroek on Monday December 26, 2016 @01:03PM (#13259108)
The concern I initially raised is just that retrofitting parametric polymorphism into a statically typed language with subtype polymorphism and a sizeable codebase worth not breaking... has a history of not going so well.

Although zero values are inelegant, I don't see why they would interact so badly with parametric polymorphism. In Rust terms, it's as if every type implicitly implemented the Default trait. (EDIT: You're right, steveklabnik. It's closer to Default than to Zero. Thanks!)

As for nil, I'd rather not comment, since I'm not familiar with the specifics of how it works in Go. (But a previous conversation with pcwalton made it perfectly clear that it doesn't work the same as in Java or .NET.)

Re:Channels in Golang by steveklabnik on Monday December 26, 2016 @02:15PM (#13259381)
(I think it's closer to Default than Zero, personally.)
Re:Channels in Golang by pcwalton on Sunday December 25, 2016 @03:48PM (#13255373)
> Aren't Go structs subtypes of every interface they conform to?

No. There's a defined coercion between values and interface types, which is not the same thing as subtyping. This affects the semantics: for example, you can't pass a value of type []MyStruct to a function taking []interface{}.

Re:Channels in Golang by catnaroek on Sunday December 25, 2016 @05:24PM (#13255699)
AFAICT, that only means that [] isn't covariant, which is only to be expected, since slices aren't immutable in Go.
Re:Channels in Golang by unscaled on Saturday December 24, 2016 @11:23PM (#13252909)
Generics are not a show stopper, since they already exist (built-in support) for everything the language creators care about.

Whether or not they are a show stopper for the users - well, that depends on your use case and preferences. Since show stoppers are a subjective thing. Objectively you could implement everything in Assembly or even Machine code, and people used to write rather complex programs this way.

I personally know there are a lot of types of systems I would never implement in Go if given the choice, because it would be too bothersome, and I don't like RSI. YMMV.

Re:Channels in Golang by closeparen on Saturday December 24, 2016 @11:35PM (#13252942)
Do we know that Google actually uses Go extensively? AFAIK the Downloads web server is the only piece of Google publicly disclosed to be written in Go.

IIRC most of Google is C++ and Java, which both have generics.

Re:Channels in Golang by dispose13432 on Sunday December 25, 2016 @12:10AM (#13253034)
I don't think they created the language for fun.

I read somewhere that they have a lot (I think it was in the MLOC range) of proprietary/in-house code written in go.

Re:Channels in Golang by mseepgood on Sunday December 25, 2016 @03:01AM (#13253425)
Re:Channels in Golang by eggie5 on Saturday December 24, 2016 @11:37PM (#13252946)
Kubernetes is written in go
Re:Channels in Golang by kasey_junk on Sunday December 25, 2016 @12:04AM (#13253017)
Is kubernetes widely used at google?
Re:Channels in Golang by lobster_johnson on Sunday December 25, 2016 @01:05AM (#13253174)
Not widely, as far as I know. Google has two existing systems, Borg and Omega, whose ideas were distilled into Kubernetes. According to an answer on Quora in May [1], Kubernetes is only used for some things that run on the Google Cloud Platform. Some more background here [2].

[1] https://www.quora.com/Does-Google-use-the-Open-Source-Kubern...

[2] http://queue.acm.org/detail.cfm?id=2898444

Re:Channels in Golang by Cyph0n on Sunday December 25, 2016 @01:20AM (#13253209)
That's pretty disappointing honestly. So which container management platform is actually used at scale?
Re:Channels in Golang by lobster_johnson on Sunday December 25, 2016 @01:28AM (#13253228)
Open source ones? Or did you mean internally within Google?

Kubernetes is growing in popularity. Among older systems, Mesos (with Marathon, usually) seems to be quite popular.

Re:Channels in Golang by Cyph0n on Sunday December 25, 2016 @02:54AM (#13253412)
No, in general. Is there a solution that is actually well tested in production? I can go through the marketing spiel claiming that the solution in question is "scalable", but I'd rather see that in practice.
Re:Channels in Golang by lobster_johnson on Sunday December 25, 2016 @03:37AM (#13253473)
Kubernetes is definitely being used at scale in production.

Spotify uses Helios [1] to run their stuff. We evaluated it briefly and decided it was too limited and immature compared to Kubernetes.

Mesos + a framework such as Marathon or Aurora was the most needy choice before Kubernetes came on the scene. Mesos probably scales farther than Kubernetes in terms of pure cluster sizes, but it also depends on what framework you use on top (Mesos itself is just a scheduler). I don't know if any of them are as flexible as Kubernetes in terms of things like volume management, config/secret management and security. It's also worth pointing out that Kubernetes can run on Mesos.

[1] https://github.com/spotify/helios

Re:Channels in Golang by bogomipz on Sunday December 25, 2016 @10:56AM (#13254456)
>"Mesos + a framework such as Marathon or Aurora was the most needy choice before Kubernetes came on the scene."

How is Mesos "needy"? Can you elaborate?

Needy has a negative connotation and it's not word I would necessarily associate with the Apache Mesos project. I've run a number of clusters in production now for just over a year with Marathon and it pretty much "just works." I have done 5 or 6 rolling upgrades now without issues. I haven't found it to be needy at all, quite the opposite, its been rock solid and the management overhead has been nominal.

>" I don't know if any of them are as flexible as Kubernetes in terms of things like volume management, config/secret management and security."

I think that Mesos is actually more flexible as it allows you to cherry pick the non-scheduler specific components to fit your use case. As an example for secrets management you can use something like Consul Vault or integrate Keywhiz or completely roll your own.

I feel like with Kubenetes you buy into the "whole thing". Using the example of secret management - you have one choice for secret management and the last time I checked they were stored in etcd in clear text. So if that doesn't fit your security requirements it seemed like you were out of luck.

Mesos also has a nice for for persistent volumes:

See: http://mesos.apache.org/documentation/latest/persistent-volu...

and:

http://schd.ws/hosted_files/mesosconeu2016/08/MesosConEurope...

Re:Channels in Golang by lobster_johnson on Sunday December 25, 2016 @02:38PM (#13255144)
> How is Mesos "needy"

Sorry, that was the damn autocorrect — I typed "beefy". As in large and powerful.

I'm not surprised if Mesos has solutions for what you describe. When I said "I don't know", I meant it literally! :-)

That said, Kubernetes provides nice built-ins, but doesn't force you to use them. You can use Vault for secrets management, for example.

Re:Channels in Golang by twic on Sunday December 25, 2016 @09:21AM (#13254155)
The container scheduling bit of Cloud Foundry (Diego aka Elastic Runtime) is used to run Pivotal Web Services, which is a public cloud thing that is reasonably big, although I don't know the actual numbers. Baidu run some big instances in-house, but again, I don't know how big.
Re:Channels in Golang by alecthomas on Sunday December 25, 2016 @05:03AM (#13253594)
Mesos was partially developed at Twitter, and was well on the way to powering most of their infrastructure when I left 3 years ago.
Re:Channels in Golang by vitus on Sunday December 25, 2016 @12:22AM (#13253085)
In short, yes. How else would we provide scalable containerized services? Docker?? [0]

[0] Don't be silly, of course we rolled our own.