News for hackers. Stuff that matters.
Re:Go 1.7 toolchain improvements by tptacek on Saturday April 02, 2016 @10:35AM (#11411661)
If you're not a Go programmer, bear this in mind: the Go compiler got noticeably slower after 1.4, but it is still extraordinarily fast; "run debounced after every keystroke" fast; "rebuild the world on a whim" fast.

Personally, I think the Go community is a little unhealthily obsessed with this particular metric.

Re:Go 1.7 toolchain improvements by dvirsky on Saturday April 02, 2016 @10:44AM (#11411699)
To me it's a broken window thing. Go's main mission is to make life easier for me as a developer, and compile times are a big factor in that. If we stopped being obsessed about it, we'd eventually end up with compile times reminiscent of C++.
Re:Go 1.7 toolchain improvements by awinter-py on Saturday April 02, 2016 @12:14PM (#11412047)
exactly. I don't understand why gerrit's stop & frisk feature is so controversial.
Re:Go 1.7 toolchain improvements by munificent on Saturday April 02, 2016 @01:23PM (#11412317)
> I think the Go community is a little unhealthily obsessed with this particular metric.

I've been on teams that are not obsessed with this metric and what very often happens is that every patch landed makes the system just a tiny bit slower. The next thing you know, your tools feel glacial and users are miserable.

And, in fact, at work right now my team is trying to feverishly claw back some performance we lost over the past few months without really noticing.

If you're working on a system where performance is a key part of the UX, you really do have to be vigilant about it.

Re:Go 1.7 toolchain improvements by vph on Saturday April 02, 2016 @11:32AM (#11411859)
>Personally, I think the Go community is a little unhealthily obsessed with this particular metric.

Let's not forget that Go was invented to solve Google's problems, one of which is it took hours to compile their C,C++ codes. Compile time of Go 1.7, despite the improvement, is still 2x that of Go 1.4.

Re:Go 1.7 toolchain improvements by pjmlp on Saturday April 02, 2016 @12:47PM (#11412173)
C was already taking hours to compile in 2000, whereas languages with modules from Algol family would compile just in several minutes.

What I find positive about Go is making younger generations that only know C and C++ rediscover the compile times we had in the native compilation during the 90's.

Re:Go 1.7 toolchain improvements by sievebrain on Saturday April 02, 2016 @02:18PM (#11412583)
Yes, in the mid nineties one of Delphi's killer features was super fast compiles. Partly possible due to a single pass compiler but it was just a very fast language to compile.

Really, most compiled languages except C++ are fast to compile. In the other thread I pointed out that Java compiles very fast, but nobody in the Go community likes to talk about that, it seems. Actually Java does have multi-tiered compilers, so the frontend is very fast as it doesn't optimise, and then the JIT compiler has a fast-but-low-quality compiler and a slower-but-higher-quality compiler, meaning you get the developer benefits of instant turnaround but the code still gets heavily optimised to GCC/LLVM quality in the hot spots.

Re:Go 1.7 toolchain improvements by colin_mccabe on Saturday April 02, 2016 @03:50PM (#11413061)
I agree that the Java compiler itself runs fast, but many of the surrounding tools in the Java ecosystem are not fast. For example, Maven takes several minutes to rebuild Hadoop. Based on my experiences, neither Gradle nor ant are that much fast than Maven (although they are a bit faster.)

I disagree that "most compiled languages... are fast to compile." Scala is relatively slow to compile (I've seen Spark take over an hour to build). OCaml and SML are none too speedy to compile. Rust's compiler is rather slow (they're going to focus on speeding it up, I have heard.) It seems like most compiled languages that people actually use are slow to compile.

Go is a big exception due to its conscious decision to focus on fast compile times. This choice wasn't free-- it involved carefully considering tradeoffs. For example, Go compiles source code files unit-by-unit, forgoing global optimizations in the name of time. The grammar of the language is simpler, in order to enable more efficient parsing. Features like operator overloading are not present. It is trivial to tell what is a function call what is not-- there is no "syntactic sugar" like Ruby's function calls without parens.

Re:Go 1.7 toolchain improvements by sievebrain on Saturday April 02, 2016 @04:40PM (#11413316)
Hadoop is 2 million lines of code. Yes, that's going to take a few minutes to compile no matter what language it's written in (if it has a compile stage at all).

By "most compiled languages" I guess I was thinking of the most popular ones, not literally all compiled languages. Sloppy phrasing, sorry. The top compiled languages according to TIOBE are Java, C, C++, C# (actually these are the top languages in general). Then they're all scripting languages until position 11 which is Delphi. So those are the top 5 and except for C/C++ they compile fast.

OCaml, SML, Haskell, Rust etc are all rather rare languages.

Re:Go 1.7 toolchain improvements by colin_mccabe on Saturday April 02, 2016 @06:00PM (#11413697)
I don't agree that 2 million lines of code is "going to take a few minutes to compile no matter what [compiled] language it's written in." Based on my experiences with Go so far, I believe Go could get that significantly faster, probably under a minute. Also, the "several minutes" number I quoted is also for incremental compiles where only one very small thing has changed, not just for full compiles, which take even longer.

TIOBE is a poor measure of programming language popularity for a lot of reasons. But even if we accept that Java, C, C++, and C# are the top languages, I already commented that Java's build is not that fast in the real world (except for very small projects). I don't have any first-hand experience with C#, but I've never heard anyone claim fast compilation is an advantage of that language. C and C++ are slow builders, as we already know. So the top compiled languages are rather slow compilers, which is one reason why scripting languages (I assume you mean non-ahead-of-time-compiled-languages) became popular in the 2000s. This is one part of what Rob Pike was talking about when he said "poorly designed static type systems drive people to dynamic typing."

Re:Go 1.7 toolchain improvements by sievebrain on Sunday April 03, 2016 @02:55PM (#11417141)
I would be skeptical of that, but I wasn't able to find LOC/sec stats for the Go compiler(s) anywhere. If you can point me to LOC/sec performance figures, that'd be interesting.

Gradle/Maven don't do incremental compilation at the level of the file, so yes, they're gonna redo the whole thing each time. Use an IDE for that. IntelliJ will happily do incremental compilation in Java projects at the level of the individual file (and there's no relinking overhead).

I understand that you're saying the top languages are slow builders, but it's just not the case. It sounds like you've used tools which simply aren't optimised for build times and judging the language based on that. But I usually have roundtrip times between editing my code and running the recompiled program of about a second when working with Java.

Re:Go 1.7 toolchain improvements by colin_mccabe on Tuesday April 05, 2016 @04:19AM (#11428729)
I have used both Eclipse and Intellij for building Hadoop, and neither could get incremental compilation under 30 seconds. In fact, it was often much more than that. Eclipse was a huge pain to use with Hadoop because it was always getting something wrong-- many times when you updated the project dependencies, Eclipse would essentially break and need manual fixing. Even loading up the project in either IDE takes at least a solid minute, with an SSD, Intel i7, and 16 GB of RAM.

Prior to using Java, I used C++ for 10 years. Every project I worked on had compile time problems. It was an open secret that we spent most of our time waiting for compiles.

It's very common for people working on small projects to feel good about their compile times. But sometimes projects grow up, and they outgrow the language and the tools they were designed in.

Re:Go 1.7 toolchain improvements by incepted on Saturday April 02, 2016 @11:44PM (#11414830)
> TIOBE is a poor measure of programming language popularity for a lot of reasons.

And yet, it is completely consistent with pretty much all other measurements of programming language popularity (github, job postings, StackOverflow answers, etc...).

So maybe it's not that poor a measure after all.

Re:Go 1.7 toolchain improvements by vorg on Sunday April 03, 2016 @05:53PM (#11417958)
According to TIOBE, Groovy rose from 0.33% to 1.8% in the last two months, and from 0.11% to 1.8% in the last 12 months. Click on "Groovy" from the main page and you'll get the graph at http://www.tiobe.com/tiobe_index?page=Groovy . Now say "maybe TIOBE's not that poor a measure of programming language popularity" with a straight face. These stats are gamed by language backers to make their language look good when selling consulting services and tickets to conferences, convincing programmers to contribute free work to their product, etc.
Re:Go 1.7 toolchain improvements by airless_cotton on Sunday April 03, 2016 @06:48AM (#11415669)
TIL that there are still people who believe that parsing has any impact on compilation speed.
Re:Go 1.7 toolchain improvements by pjmlp on Sunday April 03, 2016 @12:11PM (#11416572)
Do you know Walter Bright?

The guy that created the first C++ native compiler and one of the D authors?

http://www.drdobbs.com/cpp/c-compilation-speed/228701711

Re:Go 1.7 toolchain improvements by tptacek on Saturday April 02, 2016 @11:37AM (#11411875)
At it's slowest point, this compiler took single-digit seconds for its biggest workloads. So: I don't see how Google's C++ compile times are relevant.
Re:Go 1.7 toolchain improvements by incepted on Saturday April 02, 2016 @06:42PM (#11413893)
> Let's not forget that Go was invented to solve Google's problems

No, it was invented to solve the problem that a handful of Google engineers experienced while writing C++ at Google.

Most of Google's code base is written in Java and as such, doesn't suffer from slow compilations.

Re:Go 1.7 toolchain improvements by jasonwatkinspdx on Saturday April 02, 2016 @11:58AM (#11411962)
I suspect there's a bias born of some of the key go folks working in the context of truly leviathan single repos. It shows up in the slowness to understand what other people need re package management as well.
Re:Go 1.7 toolchain improvements by dilap on Saturday April 02, 2016 @10:53AM (#11411726)
Post 1.4 it went from "don't even think about it" to "mild annoyance".

Obviously nowhere near the pain of say C++, certainly something that was missed and will be nice to get back!

Re:Go 1.7 toolchain improvements by eternalban on Saturday April 02, 2016 @01:29PM (#11412353)
Go's dependence on source level transformations to address gaps in the language make it a reasonable requirement.
Re:Go 1.7 toolchain improvements by khyryk on Saturday April 02, 2016 @12:02PM (#11411990)
It just strikes me as amusing, since the tremendous amount of whining about how much of compilation time (compile-time) generics would add makes the issue appear important, yet here we have a 2-10x compilation time slowdown, but in this case it's suddenly totally fine -- the Go community is obsessed about the wrong things.
Re:Go 1.7 toolchain improvements by jerf on Saturday April 02, 2016 @12:30PM (#11412116)
I'm not sure how repeatedly apologizing for the slower compile time, promising that it's going to get better again in the future, putting an entirely new backend on the compiler, and making it a big focus of every single post about the last three Go versions constitutes being "suddenly totally fine".

To be hypocritical, there must be a difference between words and actions; it looks to me like "we're really concerned about compile times" has been backed up by actions, even if there was a regression. I don't recall them ever promising that there would never be a compile time regression because it was their #1 concern; what I recall them promising as their #1 concern was almost-but-not-quite total backwards compatibility, which they've maintained.

Re:Go 1.7 toolchain improvements by logingone on Saturday April 02, 2016 @12:14PM (#11412051)
but in this case it's suddenly totally fine

Not at all - it was one step back while getting on the right path, then speeding up again.

Re:Go 1.7 toolchain improvements by vanderZwan on Saturday April 02, 2016 @12:33PM (#11412122)
This is a temporary regression. I guess generics would be a permanent slow-down, and might not scale nicely for larger projects - although I do now know enough about the workings of compiler algorithms to judge that.
Re:Go 1.7 toolchain improvements by pcwalton on Saturday April 02, 2016 @02:14PM (#11412555)
I don't think generics would result in a compile time slowdown relative to the alternatives. If you don't have generics, programmers work around it with code duplication or they work around it with dynamic typing (interface{}, but also the runtime support for maps and channels fall into this category). Not coincidentally, these are the two main ways to implement generics in a compiler.
Re:Go 1.7 toolchain improvements by misframer on Saturday April 02, 2016 @12:11PM (#11412030)
There are still many things to optimize. One of the reasons to rewrite the compiler in Go was to make it easier to profile. The generics issue considers best-case compiler performance.
Re:Go 1.7 toolchain improvements by sievebrain on Saturday April 02, 2016 @02:16PM (#11412576)
Why would being written in Go make anything easier to profile?
Re:Go 1.7 toolchain improvements by wfsfw4w44 on Saturday April 02, 2016 @06:27PM (#11413822)
by being able to use the go toolchain on the go source itself (it was previously in C)