News for hackers. Stuff that matters.
Re:Proposal: Go should have generics by ktamura on Thursday April 14, 2016 @01:34AM (#11494494)
This. For better and for worse, Go was designed for "simplicity", and generics are anything but simple. I'd be very, very surprised if Go thinks about generics in earnest anytime soon.

I don't say this in anyway to eulogize Go: In some ways, Go is pathetically unexpressive. That said, it currently fills that gap for writing middleware between C sacrificing too much developer productivity and Perl/Python/Ruby/PHP sacrificing too much performance. Generics would be nice to have for this core use case for Go, but it's probably not critical.

Re:Proposal: Go should have generics by pcwalton on Thursday April 14, 2016 @01:41AM (#11494523)
Generics are a very simple feature as implemented in, for example, SML or OCaml. They're much simpler than Go interfaces, in fact.
Re:Proposal: Go should have generics by eru on Thursday April 14, 2016 @03:06AM (#11494811)
Generics become complicated when you have other subtype relationships, don't they?

(That's why SML, OCaml and Haskell don't really do inheritance, isn't it?)

Re:Proposal: Go should have generics by Intermernet on Thursday April 14, 2016 @03:49AM (#11494957)
Go doesn't do inheritance either. It has type embedding, but it's not the same.

In the most recent of Ian Lance Taylor's proposals (Type parameters, 2013 [1]) he summarizes:

> The implementation description is interesting but very complicated. Is any compiler really going to implement all that? It seems likely that any initial implementation would just use macro expansion, and unclear whether it would ever move beyond that. The result would be increased compile times and code bloat.

So I'm pretty sure that the logic is acceptable, but it conflicts with Go's core goals of simplicity and compilation speed.

[1]: https://github.com/golang/proposal/blob/master/design/15292/...

Re:Proposal: Go should have generics by masklinn on Thursday April 14, 2016 @05:34AM (#11495293)
> Go doesn't do inheritance either. It has type embedding, but it's not the same.

It does have interfaces though.

Re:Proposal: Go should have generics by Intermernet on Thursday April 14, 2016 @10:05AM (#11496773)
This is generally brought up as the reason that Go doesn't need generics.

I tend to agree with this. I've yet to come across a use-case where the current system is too difficult to deal with, but there are other people who have hit this limitation.

Maybe this can be solved by simply modifying the parsing of the keyword "type" (Or adding a reserved type "T") and telling developers that the functionality of "go generate" will automatically (in the compiler) expand and create type methods at compile time, and build the type generation into the compile phase, rather than a manual pre-compilation phase. I haven't considered the problems with this approach, but I assume Ian et al have.

It seems to me that the generated code approach could be spliced into the compiler with a few key-word parsing changes, but I'm not going to assume that the Go team haven't already thought of this, and there are probably problems with the idea that I haven't considered, above and beyond spec / compatibility promises etc.

Re:Proposal: Go should have generics by masklinn on Thursday April 14, 2016 @10:10AM (#11496821)
> This is generally brought up as the reason that Go doesn't need generics.

Which I find nonsensical, but my point was that Go's interfaces are a form of subtyping, which as eru and tel note tend to have challenging interactions with generics.

Re:Proposal: Go should have generics by loup-vaillant on Thursday April 14, 2016 @08:29AM (#11496055)
If I have generics, first class functions, and garbage collection, frankly I don't care about sub-typing.

http://loup-vaillant.fr/articles/classes-as-syntactic-sugar

Re:Proposal: Go should have generics by eru on Thursday April 14, 2016 @09:13PM (#11501438)
I like Haskell, too. But we are talking about adding generics to Go as it is. Not about designing a good language from scratch.
Re:Proposal: Go should have generics by tel on Thursday April 14, 2016 @08:03AM (#11495899)
Generics and many kinds of subtyping do cause challenges. This is part of why Scala's type inference is so bad.
Re:Proposal: Go should have generics by catfest on Thursday April 14, 2016 @01:55AM (#11494574)
I don't see why you'd choose Go instead of a JVM language like Java, you get the language simplicity (plus features like Generics) and the performance upside too.
Re:Proposal: Go should have generics by eva1984 on Thursday April 14, 2016 @03:54AM (#11494974)
Java itself is, IMHO, quite straightforward.

But setup a java toolchain, building, deploying, and a lot of other configuration if some heavy framework is involved, is non-trivial. Gradle is like a must for modern Java application, and mastering itself takes some efforts. Go, when coming to toolchain, it is pretty much battery-included, best-practice-builtin, sometimes even a little forced.

Language wise, Java recently has seem a more aggressive adoption of new and modern features, which is quite welcome for me personally, but it is still more LOC comparing to Go.

I think Go is the new Python for light to middle complexity web service, with fewer people. Java is more for mature stuff, for larger scale collaboration.

Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @06:10AM (#11495413)
A build.gradle file that lists a few dependencies is like maybe 7 or 8 lines of code, which can almost all be cargo culted. You only need to start consulting the Gradle manual once you start doing things like defining custom build tasks or wanting to use custom plugins.

Go's toolchain doesn't even bother with versioning. That's like the opposite of batteries-included, forced-best-practices. But of course it will seem simpler than a tool that does handle these basic things.

If you want the benefits of Java with a lighter syntax then look at Kotlin.

Re:Proposal: Go should have generics by vorg on Friday April 15, 2016 @07:47PM (#11508450)
> A build.gradle file that lists a few dependencies is like maybe 7 or 8 lines of code, which can almost all be cargo culted

...and that code is written in Apache Groovy. Strange why they'd bundle a Turing-complete scripting language for their build file DSL when it's only 7 or 8 lines long.

Re:Proposal: Go should have generics by twic on Thursday April 14, 2016 @03:32PM (#11499672)
> Java [...] is still more LOC comparing to Go

My experience is the exact opposite: Go takes more lines to do something than Java.

I would say that in large part, this is because the error handling restricts expressions to a rather small size, and then because without streams, collection manipulation has to be written out longhand.

Re:Proposal: Go should have generics by eva1984 on Thursday April 14, 2016 @07:14PM (#11500993)
I agree with you on the error handling part, although it is not a big pain for me yet.

But in terms of parallel programming, when doing in Java, I constantly find myself basically building a lot of stuff where Go has as a part of its own semantic. Queues -> Channel, Executors -> M in Go, and Runnables -> Go functions. Java8's Lambada and ForkJoinPool is an advance in the right direction but still not quite there.

Re:Proposal: Go should have generics by thegenius2000 on Thursday April 14, 2016 @03:38AM (#11494913)
Language simplicity? I disagree, Java is only agreable if you're comfortable with (1) being forced to work in an OOP-only environment and (2) using the JVM. And while you can argue for the upsides of both of these (which I believe are few and far between) they certainly add a great deal of clunky complexity, which many programmers are fleeing to Golang to avoid.
Re:Proposal: Go should have generics by omginternets on Thursday April 14, 2016 @03:27AM (#11494876)
>you get the language simplicity

And the most complex toolchain imaginable. This is what turns me off to Java, personally, but I think my case is fairly representative.

Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @06:14AM (#11495429)
If popular Java toolchains are the most complex you can imagine, I assume you have never encountered autotools, or really any toolchain for a large C++ project.

Toolchains normally mean build systems, debuggers, profilers, editors and other things.

Java itself doesn't require any build tool at all, you could do it all with a custom shell script. The next step up after that is an IDE like IntelliJ where you press "new project" and just start writing code. The IDE's build system will do it all for you. There is no complexity.

But most people want features like dependency management, IDE independence, command line builds, ability to customise the build with extra steps and so on. That's when you upgrade to something like Gradle (or maybe Maven if you like declarative XML). That'll give you dependency resolution with one-line-one-dependency, versioning, automatic downloads, update checking and other useful features. Many IDEs can create a Gradle project for you.

When I first encountered Java it seemed the most popular build tool was Maven, which looked very over complex at first due to its poor docs and love of inventing new words, but pretty quickly found that it wasn't so bad in reality. Gradle avoids the custom dictionary and uses a much lighter weight syntax. It's pretty good.

Re:Proposal: Go should have generics by omginternets on Thursday April 14, 2016 @08:44AM (#11496137)
>If popular Java toolchains are the most complex you can imagine, I assume you have never encountered autotools, or really any toolchain for a large C++ project.

The point was about Java, so I was responding to that, but yes, I steer clear of C++ (when possible) for the same reason.

Re:Proposal: Go should have generics by vorg on Friday April 15, 2016 @08:00PM (#11508504)
> Gradle [...] give you dependency resolution with one-line-one-dependency, versioning, automatic downloads, update checking and other useful features.

Not only automatic downloads, but also unwanted downloads courtesy of Groovy. See https://github.com/jaseg/ffi

> Many IDEs can create a Gradle project for you. Gradle avoids the custom dictionary and uses a much lighter weight syntax.

That "much lighter weight syntax" is actually all of Apache Groovy. It's grammar definition is far, far more complex than XML's.

Re:Proposal: Go should have generics by stcredzero on Thursday April 14, 2016 @07:24AM (#11495701)
The IDE's build system will do it all for you. There is no complexity.
Re:Proposal: Go should have generics by krakensden on Thursday April 14, 2016 @10:24AM (#11496963)
The siren song of just-one-more-layer.
Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @11:11AM (#11497320)
I don't see your point. If you have a collection of source files, then something must search the directory tree to find them and feed them to the compiler ... ideally, only the files that have changed, to give fast incremental compilation.

If you use a typical Java IDE like IntelliJ then the program that does that will be the IDE. There is no "one more layer" because that's the first and only layer.

If the IDE build system does not provide enough features or you'd like your codebase to be IDE independent, you can also use a separate build tool, or a combination of both (in which case the IDE will sync itself to the other build tool).

In that case there are two layers. But Go does not have any magical solution to that. There will be Go apps that need more than the little command line tool can do as well.

Re:Proposal: Go should have generics by omginternets on Thursday April 14, 2016 @11:32AM (#11497488)
>typical Java IDE like IntelliJ

So now I need to change my text editor?

Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @01:09PM (#11498513)
No. If you want to use vim then you would just use Gradle or Maven as your build system, instead of Make.
Re:Proposal: Go should have generics by omginternets on Thursday April 14, 2016 @03:45PM (#11499771)
Right, so then it is more complex than `go build`. QED.

To be clear, I'm not claiming that Go is "better"; I'm just pointing out that this is why one would chose Go over Java. Sometimes this particular benefit doesn't outweigh the costs relative to developing in Java, but language/toolchain simplicity remains -- nonetheless -- the reason why people prefer one over the other.

Re:Proposal: Go should have generics by sievebrain on Friday April 15, 2016 @05:24AM (#11503102)
Huh, no, it isn't.

Yes, "gradle build" wants to see a "build.gradle" file in the current directory, but you can run "gradle init" to get one. And after that everything except specifying dependencies is by convention.

There's really little to no difference in complexity here. What Go saves by not having a build file it loses by not encoding enough information about dependencies in the source, which leads to horrible hacks like vendoring.

Re:Proposal: Go should have generics by pythonistic on Thursday April 14, 2016 @12:11PM (#11497884)
If you're using a text editor, your IDE is likely the CLI.
Re:Proposal: Go should have generics by adrianN on Thursday April 14, 2016 @03:40AM (#11494924)
Nobody forces you to use a complex toolchain for Java. You can use javac and ed if you like. But Java is sufficiently simple and sufficiently popular for pretty awesome tooling to be available. Refactoring Java code is a breeze because your IDE understands the code perfectly.
Re:Proposal: Go should have generics by catfest on Thursday April 14, 2016 @03:31AM (#11494885)
The toolchain is what's so awesome about Java - moving to a language like Go means you lose so much, it's painful.
Re:Proposal: Go should have generics by imtringued on Thursday April 14, 2016 @02:08PM (#11499081)
Which is why it's mostly developers that previously used dynamic languages. They didn't have sophisticated IDEs.
Re:Proposal: Go should have generics by discreteevent on Thursday April 14, 2016 @03:52AM (#11494965)
A text editor, javac and java that's what I used for a few years when I started using it. I wrote a lot of code like that. I don't see why you couldn't?
Re:Proposal: Go should have generics by reycharles on Thursday April 14, 2016 @03:43AM (#11494939)
> you get the language simplicity

I remember the Go language specification to be about as long as the table of contents for the Java language specification.

On the other hand, Brainfuck is an extremely simple language, too.

Re:Proposal: Go should have generics by masklinn on Thursday April 14, 2016 @05:53AM (#11495361)
> I remember the Go language specification to be about as long as the table of contents for the Java language specification.

I'm not sure where you got that from. On my browser and screen, the JLS8 TOC[0] is 16 pages high which brings me about 20% into the Go language spec[1].

But then again that's a completely inane appeal to emotions: because it's a specification for cross-platform and cross-implementation compatibility (not a user-targeted documentation):

* the JLS is exactingly precise, the JLS's "lexical structure" section is about 3 times longer than GoSpec's "Lexical Elements", the JLS's "Execution" section is about 6 times longer than GoSpec's "Program initialization and execution"

* the JLS contains entire sections which don't get a mention in GoSpec, like binary compatibility concern, or the language's entire execution model (calling a function gets a page in gospec, it gets 20+ in the JLS) and its multithreaded memory model

The JLS is longer because its goal is that you be able to reimplement a Java compiler and runtime just from it, it's Java's entire rulebook.

Go's language spec is a much fuzzier document targeted towards language users — much like e.g. Python's language reference — there is no way you can write a clean-slate implementation just from the language spec.

[0] https://docs.oracle.com/javase/specs/jls/se8/html/index.html

[1] https://golang.org/ref/spec

Re:Proposal: Go should have generics by enneff on Thursday April 14, 2016 @08:11AM (#11495939)
> Go's language spec is a much fuzzier document targeted towards language users — much like e.g. Python's language reference — there is no way you can write a clean-slate implementation just from the language spec.

That's not correct. The Go spec is designed to be a precise specification of the language, targeted at language implementers. Ian Lance Taylor (incidentally, the author of these generics proposals) wrote gccgo based on that spec. There have been a couple of other implementations based on that spec since.

The main Go compiler itself was implemented from that spec, too. The spec comes first.

Re:Proposal: Go should have generics by reycharles on Thursday April 14, 2016 @06:21AM (#11495458)
You are absolutely right, it's a silly comparison. The Go language spec is indeed vague.

I did this comparison a while ago. It wasn't very accurate. The Go spec has probably changed. Unfortunately, it seems they don't keep older specs around(!) If I adjust the font size in the ToC of the JLS I get 23 pages and the Go Spec is 84 pages (27%). Not quite "about the same length", still.

I took a compiler course in university where we implemented a compiler for a subset of java 1.3 (I believe), and the next year I was a TA in the compiler course. I got to read the (older) JLS quite a lot. I do find Java to be a more complicated language than Go. This does not mean I find it simpler to write programs in Go (c.f. Brainfuck).

Re:Proposal: Go should have generics by enneff on Thursday April 14, 2016 @08:12AM (#11495946)
> Unfortunately, it seems they don't keep older specs around(!)

https://github.com/golang/go/commits/master/doc/go_spec.html

Re:Proposal: Go should have generics by riffraff on Thursday April 14, 2016 @02:06AM (#11494607)
Single small binaries are easy with go, very hard with java.
Re:Proposal: Go should have generics by pjmlp on Thursday April 14, 2016 @02:42AM (#11494715)
If you have the money, there are lots of commercial JVMs with compilers to native code.

There are quite a few open source ones, but they aren't as stable.

As always, Language != Implementation.

Re:Proposal: Go should have generics by thegenius2000 on Thursday April 14, 2016 @03:41AM (#11494928)
Agreed, but remember that

  Language == Thought-space >= Implementation,
so you want to maximize your language's expressiveness.
Re:Proposal: Go should have generics by fosk on Thursday April 14, 2016 @03:32AM (#11494889)
Could you recommend any that you have had experience with?
Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @07:57AM (#11495866)
RoboVM is one that compiles AOT ARM binaries, it's intended for the iPhone but it runs on MacOS too.

Avian is a JIT compiling JVM but one which is much smaller than HotSpot. It has a mode where it statically links your JAR into the binary itself, so you get a single self contained executable. With ProGuard and other optimisations like LZMA compression built in, such binaries can be remarkably small. Try their example:

https://readytalk.github.io/avian/#-xamples

It's a full blown GUI app that demonstrates the full range of widgets available, is cross platform, and yet is only about 1mb in size.

Re:Proposal: Go should have generics by bmaupin on Thursday April 14, 2016 @09:21AM (#11496401)
RoboVM can also compile to OS X and Linux:

Usually this means iOS and the ARM processor type but RoboVM is also capable of generating code for Mac OS X and Linux running on x86 CPUs. [0]

There are several forks of the latest open-source version of RoboVM. This one in particular is targeting desktop/server usage:

https://github.com/ashleyj/aura

See also: https://news.ycombinator.com/item?id=7579737

[0] http://docs.robovm.com/advanced-topics/compilation.html

Re:Proposal: Go should have generics by pjmlp on Thursday April 14, 2016 @07:32AM (#11495738)
I just know them from Java conferences.

Excelsior JET is the most well known one.

Oracle actually also supports AOT but only on embedded systems, with the commercial JDK.

Re:Proposal: Go should have generics by rat87 on Thursday April 14, 2016 @02:35AM (#11494688)
Also smaller memory usage.
Re:Proposal: Go should have generics by incepted on Thursday April 14, 2016 @02:12AM (#11494626)
> Single small binaries are easy with go

With static linking? I don't think so.

Re:Proposal: Go should have generics by dikaiosune on Thursday April 14, 2016 @02:27AM (#11494660)
Compared to a JAR with many dependencies and no option for LTO?
Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @06:19AM (#11495448)
You can make an executable "fat jar" with Capsule. It has a little shell script prepended to the JAR which means you can run it like "chmod +x foo.jar; ./foo.jar"

You can do dead code elimination and other forms of LTO using ProGuard. Just watch out for code that uses reflection. Java 9 will include a less aggressive version of the same thing which performs various link time optimisations like deleting modules (rather than individual methods/fields), statically pre-computing various tables, and converting from JAR into a more optimised (but platform specific) format.

That tool can also bundle a JRE in with your app, giving you an "tar xzvf and run" deployment model. It's not a single file, but it makes little difference in practice. The same tool can build DEBs, RPMs, Mac DMGs and Windows EXE/MSI installers with a bundled and stripped JRE too.

Re:Proposal: Go should have generics by dikaiosune on Thursday April 14, 2016 @12:38PM (#11498181)
I'm a big fan of Capsule, actually. My point was not that Java and the JVM ecosystem are terrible (I quite like them), but rather that there is a spectrum of size and complexity and that Go's static binaries seem to be on the simpler to build side of JARs and on the smaller side of JARs.

Also, I don't think there's much of a case to be made that bundling a JRE with your JAR is small, even though the tooling might be simple and it might resolve many deployment issues.

Re:Proposal: Go should have generics by pjmlp on Thursday April 14, 2016 @02:45AM (#11494730)
Java 9 brings LTO via jlink.
Re:Proposal: Go should have generics by prodigal_erik on Thursday April 14, 2016 @06:55PM (#11500918)
Putting a jar on your classpath works just like depending on a shared library but with much stronger compatibility guarantees and better chances for optimization.
Re:Proposal: Go should have generics by mappu on Thursday April 14, 2016 @02:58AM (#11494775)
Does golang have LTO? (maybe only in gccgo?)
Re:Proposal: Go should have generics by jsmthrowaway on Thursday April 14, 2016 @02:28AM (#11494665)
You think wrong. It was the default of the compiler for a long time and only requires minimal work now.
Re:Proposal: Go should have generics by danellis on Thursday April 14, 2016 @02:25AM (#11494654)
That comes with a big memory cost, though. Also huge startup time, so not suitable for command line tools.
Re:Proposal: Go should have generics by pjmlp on Thursday April 14, 2016 @02:44AM (#11494722)
Just get an AOT compiler, plenty to choose from.

Also since Java 8 with tiered compilation, I wouldn't consider a few ms a huge startup time.

Re:Proposal: Go should have generics by catfest on Thursday April 14, 2016 @03:35AM (#11494896)
I don't think startup time is an issue, and memory is definitely manageable - check out LMAX Disruptor (https://lmax-exchange.github.io/disruptor/)
Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @06:20AM (#11495452)
Java startup time is ~50msec, which for command line tools is fine.
Re:Proposal: Go should have generics by kgabis on Thursday April 14, 2016 @11:05AM (#11497279)
Unless you run them in a loop, which is not that uncommon.
Re:Proposal: Go should have generics by fauigerzigerk on Thursday April 14, 2016 @03:47AM (#11494953)
Memory usage and as a consequence of that excessive GC pauses. I'm not looking at any JVM language again before they introduce value types in a couple of years (maybe).
Re:Proposal: Go should have generics by mcculley on Thursday April 14, 2016 @05:29AM (#11495277)
I build soft real time simulation systems in Java. GC pauses haven't been a problem since 1.2 was released around 2000. Memory usage isn't a concern either for big applications, as there's not a lot of overhead in the runtime. There is the fact that one can't embed value types directly in objects, but I don't find that a problem in practice.
Re:Proposal: Go should have generics by fauigerzigerk on Thursday April 14, 2016 @05:36AM (#11495303)
Then your experience is very different from mine and that of many other people who resort to all sorts of off-heap solutions and distributing stuff across multiple VMs. I guess it depends a lot on the specific use case.
Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @06:16AM (#11495437)
You can get 10msec pauses or less with heaps >100GB with HotSpot if you tune things well and use the latest GC (G1).

If you want no GC pauses at all, ever, well, Go can't do that either. But if you are willing to pay money to Azul, you can buy a JVM that can. It also concurrently compacts the heap, which Go's GC does not.

The issue is not Java. The issue is the quality of freely available garbage collectors, which are very good, but not pauseless.

Re:Proposal: Go should have generics by fauigerzigerk on Thursday April 14, 2016 @07:22AM (#11495690)
>You can get 10msec pauses or less with heaps >100GB with HotSpot if you tune things well and use the latest GC (G1).

For what percentile of collections? I'm not wasting my time with incessant GC tuning only to delay that 5 minute stop the world pause for a bit longer. It's still going to hit eventually. For projects that might grow into that sort of heap size I use C++ (with an eye on Rust for the future).

You are right that Go is not a panacea for very large memory situations, but you can do a lot more before Go even needs that amount of memory.

The point is that languages without value types, such as Java and JavaScript, waste a huge amount of memory and generate a lot more garbage, thereby exacerbating all other related issues, including GC.

I have done quite a lot of testing for our workloads. Java memory usage is consistently two to three times higher than that of Go or C++. I'm unwilling to waste our money on that.

Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @07:36AM (#11495763)
In a properly tuned system with sufficient CPU capacity there should never be any full GC pauses with G1.

To get 10msec pause times with such huge heaps requires burning a LOT of CPU time with the standard JDK collectors because they can trade off pause latency vs CPU time.

This presentation shows tuning with 100msec as the target:

http://www.slideshare.net/HBaseCon/dev-session-7-49202969

Key points from the slides:

1. HBase setup with 100 GB heaps

2. Older collectors like CMS (still the default) sometimes take long pauses, like 5 seconds (not 5 minutes).

3. The new GC (G1) must be explicitly requested in Java 8. The plan is for it to be the default in Java 9, but switching to a new GC by default is not something to be taken lightly. G1 is, theoretically, configurably by simply setting a target pause time (lower == better latency but more CPU usage). Doing so eliminated all the long pauses, but a few collections were still 400msec (10x improvement over CMS).

4. With tuning, less than 1% of collections were over 300 msec and 60% of pauses were below the target of 100 msec.

Given that the Go collector, even the new one, isn't incremental or compacting I would be curious how effective it is with such large heaps. It seems to be that a GC that has to scan the whole 100GB every time, even if it does so in parallel, would experience staggeringly poor throughput.

Value types will certainly be a big, useful upgrade.

Re:Proposal: Go should have generics by fauigerzigerk on Thursday April 14, 2016 @08:32AM (#11496071)
>In a properly tuned system with sufficient CPU capacity there should never be any full GC pauses with G1.

So you use a language without value types that makes you pay for two or three times more memory than comparable languages, and then you spend your time re-tuning the GC every time your allocation or usage patterns change. Then you hope to never trigger a full GC that could stall the VM for many seconds (or in extreme cases that I have seen even minutes). That makes very little sense to me.

I cannot speak to the performance of the current Go GC for 100G heap sizes. I never tried it and I haven't read anything about it. It's not my language of choice for that sort of task either.

Re:Proposal: Go should have generics by NateDad on Thursday April 14, 2016 @10:14AM (#11496872)
100GB heap size in Go? How about 15ms pauses with zero tuning?

https://pbs.twimg.com/media/CWoAGeUW4AAy0-9.png:large

Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @11:03AM (#11497267)
They aren't comparable at all. Go doesn't collect incrementally and doesn't compact. So good luck collecting garbage fast enough to keep up with a heap-heavy app (which if you have a 100GB heap, your app probably is).

In other words, the issue is not pause time, it's also throughput.

And Go users do tune their GC: https://github.com/golang/go/issues/14161

Re:Proposal: Go should have generics by NateDad on Thursday April 14, 2016 @11:18AM (#11497371)
But Go programs use way less memory than Java applications... so the java application that uses 100GB might only use 40GB (or less) in Go. And there are tweaks you can make to hot code in Go to not generate garbage at all (pooling etc).
Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @11:08AM (#11497298)
Java's higher memory usage is not only related to value types, for example, in Java 8 strings always use 16 bit characters. That is fixed in Java 9. It resulted in both memory savings and speed improvements.

There are other sources of overhead too, but again - you seem to think Go has some magic solution to these problems. Since the new GC, Go is often using a heap much larger (such as twice the size) of actual live data. That's a big hit right there. And yes, you have to tune the Go GC:

https://github.com/golang/go/issues/14161

Even when Java gets value types, it won't magically slash memory usage in half. Pointer overheads are not that large.

Re:Proposal: Go should have generics by fauigerzigerk on Thursday April 14, 2016 @12:36PM (#11498166)
Value types are the main culprit behind Java's insane memory consumption. I found out through weeks of testing and benchmarking.

I mostly benchmarked a small subset of our own applications and data structures. We use a lot of strings, so I never even started to use Java's String class, only byte[].

I tried all sorts of things like representing a sorted map as two large arrays to avoid the extra Entry objects. I implemented a special purpose b-tree like data structure. I must have tried every Map implementation out there (there are many!). I stored multiple small strings in one large byte[].

The point is, in order to reduce Java's memory consumption, you must reduce the number of objects and object references. Nothing else matters much. It leads to horribly complex code and it is extremely unproductive. The most ironic thing about it is that you can't use Java's generics for any of it, because they don't work with primitives.

I also spent way too much time testing all sorts of off-heap solutions. At the end of the day, it's just not worth it. Using Go or C# (or maybe Swift if it gets a little faster) for small to medium sized heaps and C++ or Rust for the humungous sizes is a lot less work than making the JVM do something it clearly wasn't built to do.

Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @01:08PM (#11498498)
I admire your dedication to optimisation!

But I think your conclusion is not quite right. I said above that pointer overhead is only one source of Java memory consumption, and pointed to (hah) strings as another source. You replied and said no, it's all pointers, followed by "I never even started using strings". Do you see why that approach will lead to a tilted view of where the overheads are coming from?

If your application is so memory sensitive that you can't use basic data structures like maps or strings then yes, you really need to be using C++ at that point.

In theory, especially once value types are implemented, it would be possible for a Java app to have better memory usage than an equivalent C++ app, as bytecode is a lot more compact than compiled code and the JVM can do optimisations like deduplicate strings from the heap (already). Of course how much that helps depends a lot on the application in question. But the sources of gain and loss are quite complex.

Re:Proposal: Go should have generics by fauigerzigerk on Thursday April 14, 2016 @02:20PM (#11499179)
>You replied and said no, it's all pointers, followed by "I never even started using strings". Do you see why that approach will lead to a tilted view of where the overheads are coming from?

I see what you mean, but when I said that Java uses two or three times as much memory as Go or C++, I didn't include tons of UTF-16 strings either, assuming most people don't use as many strings as I do. If your baseline does include a large number of heap based Java String objects, the difference would be much greater than two or three times because strings alone would basically double Java's memory usage (or triple it if you store mostly short strings like words using the short string optimization in C++ for comparison)

>In theory, especially once value types are implemented, it would be possible for a Java app to have better memory usage than an equivalent C++ app, as bytecode is a lot more compact than compiled code

I'd love to see that theory :-) But let's say it was magic and the bytecode as well as the VM itself would require zero memory, any difference would still only be tens of MB. So it would be negligible if we're talking about heap sizes on the order of tens or hundereds of GB.

Re:Proposal: Go should have generics by sievebrain on Friday April 15, 2016 @05:22AM (#11503096)
Yes, if your heap is huge then code size savings don't matter much. I used to work with C++ apps that were routinely hundreds of megabytes in size though (giant servers that were fully statically linked). So code size isn't totally ignorable either.

WRT the theory, that depends what the heap consists of.

Imagine an app where memory usage is composed primarily of std::wstring, for example, because you want fast access to individual characters (in the BMP) and know you'll be handling multiple languages. And so your heap is made up of e.g. some sort of graph structure with many different text labels that all point to each other.

The JVM based app can benefit from three memory optimisations that are hard to do in C++ ... compressed OOPs, string deduplication and on-the-fly string encoding switches. OK, the last one is implemented in Java 9 which isn't released yet, but as we're theorising bear with me :)

Compressed OOPs let you use 32 bit pointers in a 64 bit app. Actually in this mode the pointer values are encoded in a way that let the app point to 4 billion objects not bytes, so you can use this if your heap is less than around 32 gigabytes. So if your graph structure is naturally pointer heavy in any language and, say, 20 gigabytes, you can benefit from this optimisation quite a lot.

String deduplication involves the garbage collector hashing strings and detecting duplicates as it scans the heap. As a String object points to the character array internally, that pointer can be rewritten and the duplicates collected, all in the background. If the labels on your graph structure are frequently duplicated for some reason, this gives you the benefits of a string interning scheme but without the need to code it up.

Finally the on-the-fly character set switching means strings that can be represented as Latin1 in memory are, with 16 bit characters only used if it's actually necessary. If your graph labels are mostly English but with some non-Latin1 text mixed in as well, this optimisation could benefit significantly.

Obviously, as they are both Turing complete, the C++ version of the app could implement all these optimisations itself. It could use 32 bit pointers on a 64 bit build, it could do its own string deduplication, etc. But then you have the same problem as you had with off-heap structures in Java etc: sure, you can write code that way, but it's a lot more pleasant when the compiler and runtime do it for you.

Re:Proposal: Go should have generics by boulos on Thursday April 14, 2016 @04:39AM (#11495110)
> I'd be very, very surprised if Go thinks about generics in earnest anytime soon.

I might agree if the submitter wasn't a person on the Go team trying to highlight some recent thinking ;)

Re:Proposal: Go should have generics by sievebrain on Thursday April 14, 2016 @06:08AM (#11495407)
I thought that too, but then at the end he says he has written such articles many times before over a period of years, and nothing ever came of them.
Re:Proposal: Go should have generics by slantedview on Thursday April 14, 2016 @01:40AM (#11494519)
Generics can be (relatively) simple - see C#, they just tend not to be in many implementations.
Re:Proposal: Go should have generics by bigdubs on Thursday April 14, 2016 @02:08AM (#11494617)
CLR (the C# runtime) generics are fully reified and about the most complicated and robust implementation available. It dynamically emits and loads types whenever a new generic parameter is seen for a function or type. This is very, very hard to get right and is not very performant. It also puts a lot of burden on the runtime vs. the compiler. Go expressly tries to keep the runtime as small as possible.
Re:Proposal: Go should have generics by pcwalton on Thursday April 14, 2016 @02:29AM (#11494667)
CLR-style just-in-time monomorphization is plenty performant: in fact, it's just about ideal. It's also not that difficult when you have a JIT when compared to the complexity you need for speculative optimizations of hot spots anyway.

In any case, the .NET approach isn't an option for AOT compilers like those of Go. For Go, the only reasonable option is ahead of time monomorphization, which really isn't that bad.

Re:Proposal: Go should have generics by agentS on Thursday April 14, 2016 @03:04AM (#11494805)
Monomorphization is not a reasonable option, in my opinion.

It forces the compiler to accept some truly awful running times for pathological cases. Atleast quadratic, probably exponential.

For languages that have reflection or pointer maps for GC or debug information for types, it can force large blowups in space as well. Go has all three of these.

The implementation would likely require runtime code-generation (or accept warts like Rust's "object safety").

Indeed, all of Ian's proposed implementations are polymorphic and seem to avoid each of these issues at first glance. The only advantage of a monomorphic implementation is performance, and considering the downsides, this'd be premature optimization forced by a language spec.

If its actually performance critical, I imagine it'd be easy to write a program that monomorphized a particular instantiation of the generic types. Indeed, the compiler would be free to do that itself, if it felt it would be worth it. Small, guaranteed non-pathological scenarios for instance.

Where if you guarantee monomorphization in a language spec, the compiler and all users are forced to accept the downsides in all instances, in exchange for often meaningless performance gains (example: any program that does computation then IO).

Re:Proposal: Go should have generics by pcwalton on Thursday April 14, 2016 @03:36AM (#11494901)
It's really not bad in practice. I've measured the amount of compilation time that generic instantiations take up and it's always been pretty low. Something like 20% (it's been a while, so take with a grain of salt), and that's with a naive implementation that doesn't try to optimize polymorphic code or perform ahead of time mergefunc. 20% is well within the project's demonstrated tolerance for compiler performance regressions from version to version. And you can do better with relatively simple optimizations. Generic compilation has been well-studied for decades; there are no unsolved problems here.

I would heavily advise against trying to do better than monomorphization with intensional type analysis (i.e. doing size/alignment calculations at runtime). We tried that and it was a nightmare. It didn't even save on compilation time in practice because of high constant factor overhead, IIRC.

Monomorphization is one of those things, like typechecking in ML, where the worst-case asymptotic time bounds look terrible on paper, but in practice it works out fine.

People point to C++ compilation times as a negative counterexample, but most of the compilation time here is in the parsing and typechecking, which a strongly typed generics implementation will dodge.

Re:Proposal: Go should have generics by the_mitsuhiko on Thursday April 14, 2016 @04:42AM (#11495118)
If generics were not a studied and well implemented concept I would agree. But we live in a world where this is just not the case. I would take a slightly slower compiler with generics support any day over the mess that go devolves into because of the lack of it.
Re:Proposal: Go should have generics by barrkel on Thursday April 14, 2016 @05:49AM (#11495348)
Bear in mind that .NET will use a shared instantiation when the generic arguments are normal reference types; a hidden parameter is required for static methods under this scheme, to pass the concrete type info. Monomorphization is only required when the vector of generic arguments has a distinct permutation of reference types vs value types.

This gives you the best of both worlds: memory efficient generics for the majority of instantiations, and compute efficient generics for the types most likely to benefit (like primitives).

Re:Proposal: Go should have generics by pjmlp on Thursday April 14, 2016 @02:46AM (#11494735)
.NET also has AOT compilation, which HNers seem to keep forgetting about.
Re:Proposal: Go should have generics by bad_user on Thursday April 14, 2016 @06:44AM (#11495540)
I think Go people have a very strange definition for simplicity, much like the population at large actually, but that's a shame really.

Simple means composed of a single element, not compound, unentangled, being the opposite of "complex", which of course means consisting of many different and connected parts. Instead Go people prefer the term to mean the now more popular meaning, which is "easy to understand, familiar".

I think a parallel can be drawn with another word from the English language: "free". You see, English doesn't have a word for the latin "liber" (libre, at liberty), like other romance languages have and I can name at least Italian, Spanish, French and Romanian (my own tongue). In these romance languages there's a firm distinction between libre and gratis, whereas in English there's no adjective signifying liberty without also meaning "at no monetary cost". I find that to be interesting and I bet it happened most probably because at some point in time these 2 issues were correlated.

Back to simplicity, while you can often correlate simplicity with easiness, mixing the terms is unjust because sometimes simple things aren't easy at all and sometimes easy things aren't simple at all (much like how sometimes gratis things are restricted and liberties aren't gratis). Speaking of which in my own native tongue the word for "easy" is also used to signify weightlessness (light, soft). Makes sense in a way, but you can see how language works.

And it would be a shame to lose the true meaning of "simple", just because its usage is at hand when trying to describe things that are or are not desirable. As in the end, this is how the meaning of words gets diluted and lost: because of our constant desire to make other people believe our ideas and buy our shit. So we exaggerate. Just a little, after all, we are only going to end up with a language that's ambiguous. What does it matter if "open source" has had a clear definition since OSI and that it wasn't in use before that, it's marketable dammit, so lets use it for all shit that has source-code available. Etc, etc.

And I get it, saying that generics aren't easy isn't so appealing, because that would be an acknowledgement of one's own capabilities and knowledge, being a metric relative to the one who's speaking. Whereas simple is an objective metric, with things being simple or complex irregardless of the person that's passing judgement. Still, generics are only as complex as static type systems. When you have a statically typed language, generics are a natural consequence. And if you don't add generics, then you need to add an "Object", "Any", "dynamic" or whatever you want to call it, which would be a whole in that static type system, not to mention introducing special cases (like the builtin arrays and maps) and that's complex by definition. Java did as well, introducing generics at version 5, when it was too late to do it cleanly and the result isn't nice at all ;-)

Re:Proposal: Go should have generics by possibility on Thursday April 14, 2016 @07:00AM (#11495598)
Tying your two examples together, the GPL version of free/libre is way more complex than the simple BSD version.
Re:Proposal: Go should have generics by bad_user on Thursday April 14, 2016 @07:52AM (#11495835)
True, GPL is significantly more complex, as it makes a difference between usage and distribution, not to mention it tries to prevent loopholes, like Tivoization and patent threats. Complex is not necessarily worse, of course it depends on context. I prefer APL 2 if you ask me.

But what matters in this case are that both correspond to the open source and free software definitions. For such licenses it means that there are things you can rely on. Like you know usage is OK for any purpose, including commercial ones. You know that you can share it with anybody, you know that derivate works are OK, again for any purpose, even though you can have restrictions on distribution. Etc.

For me clear definitions are important because then you immediately know what you can rely on and ambiguous language is bad because then we can't have a meaningful conversation. Are generics complicated? No. Are Java's generics complicated? Yes, but that's only because it was added later ;-)