News for hackers. Stuff that matters.
Re:Size visualization of Go executables using D3 by userbinator on Monday April 01, 2019 @10:37PM (#19550569)
That is indeed very surprising. I think the fact that Go isn't dead-slow can be attributed mainly to the sheer speed of CPUs. Looking at that code reminds me of the early days of PC compilers, especially the output of the trivial x+y-z function. The complete lack of push/pop instructions also shows a massive defect in the understanding of how the x86 architecture is supposed to be used. The icache bloat of doing that is enormous.

So return values are passed via memory, on the stack, not in registers like in most standard x86-64 calling conventions for natively compiled languages.

Wow. That's "worse than cdecl" --- which, despite passing parameters on the stack, will at least use the accumulator (and high accumulator) for return values that fit.

but makes it easier to provide good backtrace information

This seems to be a common line of thought but it goes against my belief in how tools should create efficient code --- anything intended for debugging purposes only should have zero effect on the executable when not being used, and compilers should focus on generating the most efficient code. Debugging information goes in a separate file and there you can put as much detail as you want. Don't make code generation worse, improve the debugging tools instead. The code will spend far more time, across everyone who uses it, being run than debugged.

Re:Size visualization of Go executables using D3 by drej on Tuesday April 02, 2019 @04:03AM (#19551997)
I wouldn't call Go dead slow, it really does depend on the use case. Go was originally designed for systems that don't require absolute max performance, it was focused on providing safety and a good standard library for building things.

You can see from the common use cases, original authors' backgrounds or issue discussions, that e.g. the target audience are not people who want maximum throughput in data processing systems (something I've been interested in). The serde libraries are dead slow (but correct), there is little to no assembly in this code (unlike in the crypto packages), you don't have any higher level access to intrinsics to build this yourself, there is no native (meaning -march) compilation (by design, for portability reasons) etc. etc.

If you try writing high performance Go, it often starts looking like C rather than Go (by avoiding channels, io.Reader, using unsafe etc). It's a shame, but oftentimes, it's your only option. Plus, you don't have clang/gcc developers to help speed up your code on a daily basis, you "only" have the Go team and contributors (yes, there is gccgo, but...).

All that considered, I like the language.

Re:Size visualization of Go executables using D3 by userbinator on Tuesday April 02, 2019 @07:16AM (#19552871)
I wouldn't call Go dead slow

I'm not saying it is -- but rather, that they could get away with such sloppy code generation because CPUs are so fast now. I did mention that early in the history of the PC, pretty much all compilers were like that due to other constraints, and the difference between that and handwritten Asm was enormous.

Re:Size visualization of Go executables using D3 by drej on Tuesday April 02, 2019 @08:10AM (#19553160)
Whoops, apologies, misread that one sentence.

Agreed - if one takes the performance on an absolute scale, it is usually sufficient (also the reason why people use Python or PHP, despite both being, relatively, quite slow).

It is only once people start comparing it to the next best thing and/or when they desire better performance that they realise, that there is a lot of not so low hanging fruit.

Re:Size visualization of Go executables using D3 by coldtea on Tuesday April 02, 2019 @11:06AM (#19554879)
>Go was originally designed for systems that don't require absolute max performance

Originally it was supposed to be a "systems" language, and was expected to lure C++ programmers.

Re:Size visualization of Go executables using D3 by j16sdiz on Tuesday April 02, 2019 @06:04AM (#19552497)
It is not as slow as it sounds.

golang do heavy inline-ing and most of its performance comes from concurrency. Putting values on stack actually make concurrency easier.

That said, there is a proposal to do it in register: https://github.com/golang/go/issues/18597 It is challenging and not in high priority. The current priority is do more inlining.

Re:Size visualization of Go executables using D3 by 1_000_000 on Tuesday April 02, 2019 @12:08PM (#19555610)
> "It is not as slow as it sounds."

Yes, it is. 5%-10% in the very link you posted.

> "golang do heavy inline-ing"

No, it does not. See https://github.com/golang/go/issues/17566

> "most of its performance comes from concurrency"

That's just ridiculous. They didn't write all crypto and math in assembly just for nothing. Lumping bunch of slow function calls onto several threads is not where performance magically comes from. And you are confusing concurrency and parallelism.

Re:Size visualization of Go executables using D3 by pcwalton on Tuesday April 02, 2019 @10:35AM (#19554497)
> Putting values on stack actually make concurrency easier.

Could you elaborate how?

Re:Size visualization of Go executables using D3 by pcwalton on Tuesday April 02, 2019 @10:24AM (#19554366)
> The complete lack of push/pop instructions also shows a massive defect in the understanding of how the x86 architecture is supposed to be used.

Push and pop aren't that great because they mutate the stack pointer for each value you push or pop instead of mutating it once. Usually you want to push or pop more than one argument at a time, or reuse outgoing stack space for multiple function calls, and in that case it's often better to do the stack pointer manipulation once and then use mov to move the values into place.

That said, passing all arguments and return values via the stack is totally silly, is a significant performance hit, and is a bug they really should fix.

Re:Size visualization of Go executables using D3 by userbinator on Tuesday April 02, 2019 @09:45PM (#19560145)
Push and pop aren't that great because they mutate the stack pointer for each value you push or pop instead of mutating it once.

The push and pop instructions are specifically optimised; see the section titled "Stack Engine" here for a brief explanation:

https://en.wikipedia.org/wiki/Stack_register

Not only does using mov with a base+displacement addressing mode not take advantage of the specialised stack hardware and also increases the code size, it also means extra effective-address calculations for the ALU.

If you don't believe that, look at this:

https://lists.llvm.org/pipermail/llvm-dev/2014-December/0799...

When someone who works for Intel proposes changing LLVM to generate push instead of mov for calls, and even suggests push-pop pairs for memory-memory moves to save on register allocation (I've seen this trick before in handwritten Asm, but can't recall if ICC does it too --- it might), I think you had better listen to him, unless you believe it's some sort of a weird conspiracy to make other compilers worse so that Intel's looks better... but then again, ICC and MSVC (among others) use push too, and they're certainly not any slower for it.

Re:Size visualization of Go executables using D3 by pcwalton on Tuesday April 02, 2019 @10:16PM (#19560260)
> If you don't believe that, look at this:

The followups cite Agner's tables to show that push and pop are slower. Reg/mem move has reciprocal throughput 0.5 on Haswell, while pop has 1.

It's a code size win, but not really a speed win.

Re:Size visualization of Go executables using D3 by senozhatsky on Tuesday April 02, 2019 @10:16PM (#19560261)
> That is indeed very surprising. I think the fact that Go isn't dead-slow can be attributed mainly to the sheer speed of CPUs.

Well, I suspect that in any reasonably complex C/C++ application (e.g. firefox) compilers do pass params via stack and probably they do so rather often. Even on x86_64.

/usr/lib/firefox/firefox objdump:

        a8b8:       48 83 ec 40             sub    $0x40,%rsp
        a8bc:       48 89 d3                mov    %rdx,%rbx
        a8bf:       48 89 f5                mov    %rsi,%rbp
        a8c2:       49 89 ff                mov    %rdi,%r15
        a8c5:       64 48 8b 04 25 28 00    mov    %fs:0x28,%rax
        a8cc:       00 00 
        a8ce:       48 89 44 24 38          mov    %rax,0x38(%rsp)
        a8d3:       44 8b 76 10             mov    0x10(%rsi),%r14d
        a8d7:       44 8b 62 10             mov    0x10(%rdx),%r12d
        a8db:       48 89 74 24 20          mov    %rsi,0x20(%rsp)
        a8e0:       48 89 54 24 28          mov    %rdx,0x28(%rsp)
        a8e5:       c7 44 24 30 02 00 00    movl   $0x2,0x30(%rsp)
        a8ec:       00 
        a8ed:       48 8d 7c 24 20          lea    0x20(%rsp),%rdi
        a8f2:       e8 39 fd ff ff          callq  a630 <_ZN7mozilla11Compression3LZ417decompressPartialEPKcmPcmPm@@Base+0x2c0>
    --
      etc.
/usr/lib/firefox/libxul.so objdump:

      81cf5e:       48 89 44 24 30          mov    %rax,0x30(%rsp)
      81cf63:       4c 89 7c 24 58          mov    %r15,0x58(%rsp)
      81cf68:       4d 89 cf                mov    %r9,%r15
      81cf6b:       f2 44 0f 11 64 24 60    movsd  %xmm12,0x60(%rsp)
      81cf72:       e8 a9 00 00 00          callq  81d020 <mont_mulf_noconv@@xul66+0xa30>
    --
      etc.
Re:Size visualization of Go executables using D3 by Thaxll on Tuesday April 02, 2019 @09:57AM (#19554094)
Go is faster than most languages so I'm not sure it's related to the recent CPU since every languages are benchmarks against the same CPU.

"The complete lack of push/pop instructions also shows a massive defect in the understanding of how the x86 architecture is supposed to be used."

I'm sure the Go team has very skilled people in that field, so I'm not sure what you mean.

You have to understand that the Go compiler does not look for the best optimized code, it's a balance between performance / compile speed and "debuggability".

Re:Size visualization of Go executables using D3 by pcwalton on Tuesday April 02, 2019 @10:32AM (#19554459)
> You have to understand that the Go compiler does not look for the best optimized code, it's a balance between performance / compile speed and "debuggability".

Passing arguments in registers doesn't help "debuggability". There is a debugging advantage to not using registers at all (though there really shouldn't be; spilling all locals to stack for -g -O0 is only necessary due to design problems in LLVM). But there's no debugging advantage to having a calling convention that spills everything on stack. If you your debugging infrastructure can handle locals in registers at all, it can easily handle arguments in registers too.

Re:Size visualization of Go executables using D3 by coldtea on Tuesday April 02, 2019 @11:09AM (#19554919)
>I'm sure the Go team has very skilled people in that field, so I'm not sure what you mean.

That's argument by authority. Go team has important industry contributors but hardly has experts that have kept up with 30+ years of PL research.

Re:Size visualization of Go executables using D3 by grumpydba on Tuesday April 02, 2019 @11:41AM (#19555319)
The team has world class compiler and GC experts. You should know better.
Re:Size visualization of Go executables using D3 by coldtea on Tuesday April 02, 2019 @12:05PM (#19555586)
Who do you mean?

People who have wrote some successful language are not necessarily "world class compiler and GC experts", and it's even less correlated with "having kept up with the last 30+ years of PL research" as mentioned above.

Would you call e.g. Kernighan or Guido "world class compiler and GC experts", compared to people like Anders Hejlsberg, Lars Bak, Martin Odersky, Wirth, Simon Peyton Jones, Lattner, and co, who are experts devoted to exactly PL (and don't miss real-world accolades).

Robert Griesemer might come close, but he doesn't appear to be the driving force.

Even more importantly, does Go strike you as the product of "PL expertise", or just a competent, pragmatic, if humble, compiler, the likes of which are many (and even more full featured, even from smaller teams)?

What I see Go having more is adoption and libs (probably due to Google's full-time dev support and branding). Not some PL upper hand over other languages like Rust, Crystal, Zig, Nim, etc.

Re:Size visualization of Go executables using D3 by Thaxll on Tuesday April 02, 2019 @12:46PM (#19556108)
What exactly from other small team have:

- dead simple cross compiling

- multi platforms / arch ( there are many for Go: https://gist.github.com/asukakenji/f15ba7e588ac42795f421b48b... )

- no dependencies

- fast compilation

- good performance

- good debugging

- rock solid and production ready

I don't know any compiler from smaller team that matches the Go one, it's one of the reason why people use Go, compiling things is fast and easy, from my raspbery pi I can compile a win10 64bits exec with 0 problems 0 tools to install. ( ex: GOOS=windows GOARCH=amd64 go build . )

Re:Size visualization of Go executables using D3 by coldtea on Tuesday April 02, 2019 @02:32PM (#19557173)
None of those are arguments for "PL experts" as none of those are specifically signs of superior PL expertise.

Multiple platforms for example are mainly about adoption (and not having much optimizations/assembly parts in the codebase, making it easier to port).

Fast compilation is something several languages manage. And being fast to compile because you don't do much (in the way of optimizations) is something most languages can manage.

"No dependencies" is also about adoption and resources (to replace popular dependencies). Nothing particular related to PL/compiler expertise about it.

Go never had a "good debugging" story, and it doesn't have the best performance either (e.g. compare with Rust, D, Crystal, etc). In certain areas like text processing it's even worse.

Re:Size visualization of Go executables using D3 by foldr on Wednesday April 03, 2019 @12:22PM (#19564674)
> none of those are specifically signs of superior PL expertise.

If none of the things on that list is a sign of superior PL expertise, one would have to ask whether PL expertise is actually worth very much.

Re:Size visualization of Go executables using D3 by grumpydba on Tuesday April 02, 2019 @04:39PM (#19558351)
Richard Hudson, Ian Lance Taylor, David Chase, Austin Clements. The team is incredibly skilled.

And of course Pike, Griesmer & Cox.

I think that they are well aware of the trade-offs they make. And I also think that it is quite arrogant to say otherwise just because your tastes are different.

Go ahead, please write the go haters' handbook. Or write rants saying 'go is obsolete'. And the future will say who has a viable legacy.

Re:Size visualization of Go executables using D3 by ngrilly on Tuesday April 02, 2019 @03:57AM (#19551966)
> anything intended for debugging purposes only should have zero effect on the executable when not being used

It's not always possible. And debugging is very important in production (be able to inspect memory usage, generate meaningful stack trace, etc.).

Re:Size visualization of Go executables using D3 by cube2222 on Tuesday April 02, 2019 @01:39AM (#19551354)
Do you have somewhere one could read up on why those really are such antipatterns?
Re:Size visualization of Go executables using D3 by nothrabannosir on Tuesday April 02, 2019 @05:03AM (#19552254)
The same author, Raphael Poss, has another post on it:

https://science.raphael.poss.name/go-calling-convention-x86-...

It's very extensive and informative.