And make the standard library unmagical? One of the things I've always disliked about Go is how stdlib provides magic facilities unavailable to user code.

I agree, but there's not much, and it's usually for decent reasons:

* the "time" package hooks into the Go scheduler.

* the "reflect" package needs type information out of the runtime.

* the "net" package hooks into the Go scheduler a bit. But way less than it used to. I think you might even be able to do this all yourself outside of std nowadays.

What else?

Our rough plan going forward is to actually move a bunch of the stdlib elsewhere but ship copies/caches of tagged versions with Go releases. So maybe "encoding/json" is really an alias for "golang.org/std/encoding/json" and Go 1.13.0 ships with version 1.13.0 of golang.org/std/encoding/json. But if you need a fix, feature, or optimization sooner than 6 months when 1.14.0 comes out, you can update earlier.

My guess that quotemstr meant builtins: append, copy, range "protocol", etc.

There are many examples of things builtins can do which user-code can't:

1. The constructs like "x <- chan" and "x, ok <- chan" (and map indexing and so on) are only available to builtin types.

It's impossible to write a thing like "x, ok <- chan" where it either panics or doesn't depending on if the caller wanted two or one return values.

2. generic types, like 'map' and 'chan'.

3. 'len' can't be implemented for a user type, e.g. "len(myCustomWorkQueue)" doesn't work.

4. range working without going through weird hoops (like returning a channel and keeping a goroutine around to feed it)

5. Reserving large blocks of memory up front (e.g. as 'make' can do for slices and maps) in an efficient manner (e.g. make(myCustomWorkQueue, 0, 20) doesn't work)

You're talking about the language. He's talking about the standard library.

We're both talking about both. The core of my complaint is that the language privileges parts of the standard library.

> privileges parts of the standard library

It really doesn't. Maps and channels are not part of the stdlib. They're part of the language, but not the library.

Similarly, range is part of the language, not the stdlib.

If the stdlib was privileged, "container/*" in it would be generic, but it's not.

All right, but my point stands. Maps and channels should be part of stdlib, and they should rely on primitives available to user code too.

Maps and channels are not part of the standard library. How you know that is, you can replace the entire standard library, using none of it, and still be programming in Go with maps and channels.

Parent wrote "should be" not "are".

Such primitives would bring a lot of complexity which is something Go tried to avoid.

Consider that make, map, chan etc are just language features/specs just like int64, string, func, main etc

Why should string be fundamental?

Nobody said it "should" be. It simply is, in Go, the language we're discussing. If you nerdsnipe an HN thread with an argument that there's no difference between a language, its runtime, and its standard library, people are going to pile on to point out that's not true.

Could, sure. Should? I think you are asserting facts that are not in evidence.

I'm a bit confused, and I'd like to understand your position better. I can see why arrays, structs/tuples, or [tagged] unions are usually primitives – they sort of express different fundamental layouts of data in memory, so they need support from the runtime/compiler. In the case of Go, channels also make sense as a primitive – as I understand it, `select` requires scheduler support.

But higher level data structures like lists, sets, maps etc. are all implemented using those primitives under the hood. So, especially in a "systems language", I would sort of expect them to just be libraries in the stdlib, not language features[1]. But if you think otherwise (or see a gap in my reasoning) I'd love to hear why!

[1] Except maybe some syntactic sugar for list/dict/set literals.

"Should" implies some normative assignment of value.

> But higher level data structures like lists, sets, maps etc. are all implemented using those primitives under the hood. So, especially in a "systems language", I would sort of expect them to just be libraries in the stdlib, not language features[1].

Maybe! But if those higher order structures are closer to language features than libraries -- and, specifically to this conversation, if they leverage parts of the runtime that aren't available to mere mortals -- is it a strictly worse outcome? I don't know. Some evidence might suggest yes. But I think it's at best arguable.

I think I understand, thanks for explaining.

> But higher level data structures like lists, sets, maps etc. are all implemented using those primitives under the hood

In Go you don't have generics as a primitive with which to build those things.

As a result, you don't have the building blocks for anyone to build a usable data structure unless it's baked into the core of the language.

I'm aware of that, and I read the generics proposal linked. I commented here because I wanted to respond to this:

> Should [be just libraries]? I think you are asserting facts that are not in evidence.

I tried to describe why I feel that go's collections should be libraries and not builtins, and hopefully understand why GP seemed like they were questioning that idea.

What's magical about the stdlib?

Maybe they meant the language, which has some generic functions like append or delete - instead of those functions being attached to a data type or accepting any type, they only act on built-in types. I doubt any of that would change though given the stated goal of remaining compatible in most cases.

There are a handful of packages (time and some stuff in os for example) that are tied into the runtime and can't be rewritten outside the standard library. It's not a huge amount, and some of it can probably be fixed in the future.

What if generics were just implemented internally in Go? IMO, there are good use cases, like generic numbers (int32, int64, etc.), list processing, etc. There's already precedence for this with map[A]B, make(), etc.

What changed? Why is the Go team finally considering this now, when it has been requested for these exact reasons by many credible developers since Go's beginning?

EDIT: Yes, something has changed. They always said "no, but maybe later" and now it's "ok how about now". That is a big change and I am just asking what's the reason for it.

A year ago we said it was time to move toward Go 2: https://blog.golang.org/toward-go2 . This is another step in that direction.

Then I'm a year late in asking why the Go team is now deciding to say yes to something they said no to for over half a decade.

The blog post explains that.

Nothing changed. It wasn't off the table to begin with. See: https://golang.org/doc/faq#generics

The question of the parent boils down to "what put it on the menu".

So, yeah, it was always "on the table" in some hazy, we'll see later, way, but not like that.

I think a good faith reading of the circumstance would lead one to conclude that the passage of time, and other, higher-priority and lower-cost-of-implementation issues being addressed, have put these issues on the menu.

Likely because they have now figured out implementations of these features which they feel work well with the rest of the language design.

Because now is later than when they said later?

The reason for saying "no, but maybe later" originally was that it was always a desired piece of functionality. I believe the word "inevitable" was even tossed around. The problem was that they didn't like the existing implementations of generics in other languages and also didn't know how they would improve it. I haven't had time to look at the details, so I'm not sure what they actually did, but I think they basically came up with a strategy that they are mostly happy with.

You might ask, why not pick a solution earlier and then improve it over time? However with a popular language it's not that easy. People are writing lots of code. If you start early, you can paint yourself in a corner and end up with a system that you really can't improve very much because it introduces incompatible changes. Note the extremely painful (and drawn out!) incompatible changes with Perl 6 and Python 3. If you make the wrong choice early, you might still take over a decade to find an opportunity to replace it. And since the state of the art in language design moves very slowly, it took a long time before they could see something that they felt they wouldn't regret choosing.

I highly recommend reading https://go.googlesource.com/proposal/+/master/design/go2draf...

This is part of the material linked to from this thread. It answers all your questions. You will see that some of the people who are directly replying to you are very much involved in this effort.

Edit: I tried editing the previous message but HN actually automatically made a reply instead....

Adding generics is a major change (e.g. it would restructure the stdlib), and so it would only belong in a major-version bump of the language.

Though the authors have clarified that the "2" in "Go 2" is just for marketing purposes, and that the actual release would still be "Go 1.x", and shouldn't break any existing code (so it won't restructure the stdlib in any major way, unless they want to do a huge mass-deprecation).

So Go is following the exact same path as Java.

Inevitably.

Also inevitably, Java will get value types (structs) at some point.

Just like with Go Generics, people will complain the feature is bolted on. After some time it is just one of the many quirks of a complex language.

I think it was Stroustrup who said: there are two kinds of programming languages, the ones people hate and the ones nobody uses.

And Java had quite a few good examples to take on like Modula-3 and Eiffel, but alas they wanted something for average Joe, kind of sounds familiar.

The difference is that Go could have learned from Java.

I think because it would break backward compatability in v1, and not doing that was the Most Important Thing in all 1.xx releases.

v2 is allowed to Break Stuff. So obviously they're going to consider the thing that people have shouted for most in Go... generics and error handling (and package management but they found a way of doing that earlier).

> What changed? Why is the Go team finally considering this now, when it has been requested for these exact reasons by many credible developers since Go's beginning?

Simply, Go has probably accumulated all the developers it is going to accumulate without something drastically changing.

At this point, I can't think of any programmers around me who now want to learn Go.