News for hackers. Stuff that matters.
Re:Size visualization of Go executables using D3 by cakoose on Monday April 01, 2019 @10:12PM (#19550432)
> as discussed in my previous article [1], Go uses memory instead of registers to pass arguments and return values across function calls.

[1] https://science.raphael.poss.name/go-calling-convention-x86-...

This is very surprising for a language that targets somewhat high performance.

Looks like it's a 5-10% performance hit, but makes it easier to provide good backtrace information: https://github.com/golang/go/issues/18597

Re:Size visualization of Go executables using D3 by userbinator on Monday April 01, 2019 @10:37PM (#19550569)
That is indeed very surprising. I think the fact that Go isn't dead-slow can be attributed mainly to the sheer speed of CPUs. Looking at that code reminds me of the early days of PC compilers, especially the output of the trivial x+y-z function. The complete lack of push/pop instructions also shows a massive defect in the understanding of how the x86 architecture is supposed to be used. The icache bloat of doing that is enormous.

So return values are passed via memory, on the stack, not in registers like in most standard x86-64 calling conventions for natively compiled languages.

Wow. That's "worse than cdecl" --- which, despite passing parameters on the stack, will at least use the accumulator (and high accumulator) for return values that fit.

but makes it easier to provide good backtrace information

This seems to be a common line of thought but it goes against my belief in how tools should create efficient code --- anything intended for debugging purposes only should have zero effect on the executable when not being used, and compilers should focus on generating the most efficient code. Debugging information goes in a separate file and there you can put as much detail as you want. Don't make code generation worse, improve the debugging tools instead. The code will spend far more time, across everyone who uses it, being run than debugged.

Re:Size visualization of Go executables using D3 by drej on Tuesday April 02, 2019 @04:03AM (#19551997)
I wouldn't call Go dead slow, it really does depend on the use case. Go was originally designed for systems that don't require absolute max performance, it was focused on providing safety and a good standard library for building things.

You can see from the common use cases, original authors' backgrounds or issue discussions, that e.g. the target audience are not people who want maximum throughput in data processing systems (something I've been interested in). The serde libraries are dead slow (but correct), there is little to no assembly in this code (unlike in the crypto packages), you don't have any higher level access to intrinsics to build this yourself, there is no native (meaning -march) compilation (by design, for portability reasons) etc. etc.

If you try writing high performance Go, it often starts looking like C rather than Go (by avoiding channels, io.Reader, using unsafe etc). It's a shame, but oftentimes, it's your only option. Plus, you don't have clang/gcc developers to help speed up your code on a daily basis, you "only" have the Go team and contributors (yes, there is gccgo, but...).

All that considered, I like the language.

Re:Size visualization of Go executables using D3 by userbinator on Tuesday April 02, 2019 @07:16AM (#19552871)
I wouldn't call Go dead slow

I'm not saying it is -- but rather, that they could get away with such sloppy code generation because CPUs are so fast now. I did mention that early in the history of the PC, pretty much all compilers were like that due to other constraints, and the difference between that and handwritten Asm was enormous.

Re:Size visualization of Go executables using D3 by drej on Tuesday April 02, 2019 @08:10AM (#19553160)
Whoops, apologies, misread that one sentence.

Agreed - if one takes the performance on an absolute scale, it is usually sufficient (also the reason why people use Python or PHP, despite both being, relatively, quite slow).

It is only once people start comparing it to the next best thing and/or when they desire better performance that they realise, that there is a lot of not so low hanging fruit.

Re:Size visualization of Go executables using D3 by coldtea on Tuesday April 02, 2019 @11:06AM (#19554879)
>Go was originally designed for systems that don't require absolute max performance

Originally it was supposed to be a "systems" language, and was expected to lure C++ programmers.

Re:Size visualization of Go executables using D3 by j16sdiz on Tuesday April 02, 2019 @06:04AM (#19552497)
It is not as slow as it sounds.

golang do heavy inline-ing and most of its performance comes from concurrency. Putting values on stack actually make concurrency easier.

That said, there is a proposal to do it in register: https://github.com/golang/go/issues/18597 It is challenging and not in high priority. The current priority is do more inlining.

Re:Size visualization of Go executables using D3 by 1_000_000 on Tuesday April 02, 2019 @12:08PM (#19555610)
> "It is not as slow as it sounds."

Yes, it is. 5%-10% in the very link you posted.

> "golang do heavy inline-ing"

No, it does not. See https://github.com/golang/go/issues/17566

> "most of its performance comes from concurrency"

That's just ridiculous. They didn't write all crypto and math in assembly just for nothing. Lumping bunch of slow function calls onto several threads is not where performance magically comes from. And you are confusing concurrency and parallelism.

Re:Size visualization of Go executables using D3 by pcwalton on Tuesday April 02, 2019 @10:35AM (#19554497)
> Putting values on stack actually make concurrency easier.

Could you elaborate how?

Re:Size visualization of Go executables using D3 by pcwalton on Tuesday April 02, 2019 @10:24AM (#19554366)
> The complete lack of push/pop instructions also shows a massive defect in the understanding of how the x86 architecture is supposed to be used.

Push and pop aren't that great because they mutate the stack pointer for each value you push or pop instead of mutating it once. Usually you want to push or pop more than one argument at a time, or reuse outgoing stack space for multiple function calls, and in that case it's often better to do the stack pointer manipulation once and then use mov to move the values into place.

That said, passing all arguments and return values via the stack is totally silly, is a significant performance hit, and is a bug they really should fix.

Re:Size visualization of Go executables using D3 by userbinator on Tuesday April 02, 2019 @09:45PM (#19560145)
Push and pop aren't that great because they mutate the stack pointer for each value you push or pop instead of mutating it once.

The push and pop instructions are specifically optimised; see the section titled "Stack Engine" here for a brief explanation:

https://en.wikipedia.org/wiki/Stack_register

Not only does using mov with a base+displacement addressing mode not take advantage of the specialised stack hardware and also increases the code size, it also means extra effective-address calculations for the ALU.

If you don't believe that, look at this:

https://lists.llvm.org/pipermail/llvm-dev/2014-December/0799...

When someone who works for Intel proposes changing LLVM to generate push instead of mov for calls, and even suggests push-pop pairs for memory-memory moves to save on register allocation (I've seen this trick before in handwritten Asm, but can't recall if ICC does it too --- it might), I think you had better listen to him, unless you believe it's some sort of a weird conspiracy to make other compilers worse so that Intel's looks better... but then again, ICC and MSVC (among others) use push too, and they're certainly not any slower for it.

Re:Size visualization of Go executables using D3 by pcwalton on Tuesday April 02, 2019 @10:16PM (#19560260)
> If you don't believe that, look at this:

The followups cite Agner's tables to show that push and pop are slower. Reg/mem move has reciprocal throughput 0.5 on Haswell, while pop has 1.

It's a code size win, but not really a speed win.

Re:Size visualization of Go executables using D3 by senozhatsky on Tuesday April 02, 2019 @10:16PM (#19560261)
> That is indeed very surprising. I think the fact that Go isn't dead-slow can be attributed mainly to the sheer speed of CPUs.

Well, I suspect that in any reasonably complex C/C++ application (e.g. firefox) compilers do pass params via stack and probably they do so rather often. Even on x86_64.

/usr/lib/firefox/firefox objdump:

        a8b8:       48 83 ec 40             sub    $0x40,%rsp
        a8bc:       48 89 d3                mov    %rdx,%rbx
        a8bf:       48 89 f5                mov    %rsi,%rbp
        a8c2:       49 89 ff                mov    %rdi,%r15
        a8c5:       64 48 8b 04 25 28 00    mov    %fs:0x28,%rax
        a8cc:       00 00 
        a8ce:       48 89 44 24 38          mov    %rax,0x38(%rsp)
        a8d3:       44 8b 76 10             mov    0x10(%rsi),%r14d
        a8d7:       44 8b 62 10             mov    0x10(%rdx),%r12d
        a8db:       48 89 74 24 20          mov    %rsi,0x20(%rsp)
        a8e0:       48 89 54 24 28          mov    %rdx,0x28(%rsp)
        a8e5:       c7 44 24 30 02 00 00    movl   $0x2,0x30(%rsp)
        a8ec:       00 
        a8ed:       48 8d 7c 24 20          lea    0x20(%rsp),%rdi
        a8f2:       e8 39 fd ff ff          callq  a630 <_ZN7mozilla11Compression3LZ417decompressPartialEPKcmPcmPm@@Base+0x2c0>
    --
      etc.
/usr/lib/firefox/libxul.so objdump:

      81cf5e:       48 89 44 24 30          mov    %rax,0x30(%rsp)
      81cf63:       4c 89 7c 24 58          mov    %r15,0x58(%rsp)
      81cf68:       4d 89 cf                mov    %r9,%r15
      81cf6b:       f2 44 0f 11 64 24 60    movsd  %xmm12,0x60(%rsp)
      81cf72:       e8 a9 00 00 00          callq  81d020 <mont_mulf_noconv@@xul66+0xa30>
    --
      etc.
Re:Size visualization of Go executables using D3 by Thaxll on Tuesday April 02, 2019 @09:57AM (#19554094)
Go is faster than most languages so I'm not sure it's related to the recent CPU since every languages are benchmarks against the same CPU.

"The complete lack of push/pop instructions also shows a massive defect in the understanding of how the x86 architecture is supposed to be used."

I'm sure the Go team has very skilled people in that field, so I'm not sure what you mean.

You have to understand that the Go compiler does not look for the best optimized code, it's a balance between performance / compile speed and "debuggability".

Re:Size visualization of Go executables using D3 by pcwalton on Tuesday April 02, 2019 @10:32AM (#19554459)
> You have to understand that the Go compiler does not look for the best optimized code, it's a balance between performance / compile speed and "debuggability".

Passing arguments in registers doesn't help "debuggability". There is a debugging advantage to not using registers at all (though there really shouldn't be; spilling all locals to stack for -g -O0 is only necessary due to design problems in LLVM). But there's no debugging advantage to having a calling convention that spills everything on stack. If you your debugging infrastructure can handle locals in registers at all, it can easily handle arguments in registers too.

Re:Size visualization of Go executables using D3 by coldtea on Tuesday April 02, 2019 @11:09AM (#19554919)
>I'm sure the Go team has very skilled people in that field, so I'm not sure what you mean.

That's argument by authority. Go team has important industry contributors but hardly has experts that have kept up with 30+ years of PL research.

Re:Size visualization of Go executables using D3 by grumpydba on Tuesday April 02, 2019 @11:41AM (#19555319)
The team has world class compiler and GC experts. You should know better.
Re:Size visualization of Go executables using D3 by coldtea on Tuesday April 02, 2019 @12:05PM (#19555586)
Who do you mean?

People who have wrote some successful language are not necessarily "world class compiler and GC experts", and it's even less correlated with "having kept up with the last 30+ years of PL research" as mentioned above.

Would you call e.g. Kernighan or Guido "world class compiler and GC experts", compared to people like Anders Hejlsberg, Lars Bak, Martin Odersky, Wirth, Simon Peyton Jones, Lattner, and co, who are experts devoted to exactly PL (and don't miss real-world accolades).

Robert Griesemer might come close, but he doesn't appear to be the driving force.

Even more importantly, does Go strike you as the product of "PL expertise", or just a competent, pragmatic, if humble, compiler, the likes of which are many (and even more full featured, even from smaller teams)?

What I see Go having more is adoption and libs (probably due to Google's full-time dev support and branding). Not some PL upper hand over other languages like Rust, Crystal, Zig, Nim, etc.

Re:Size visualization of Go executables using D3 by Thaxll on Tuesday April 02, 2019 @12:46PM (#19556108)
What exactly from other small team have:

- dead simple cross compiling

- multi platforms / arch ( there are many for Go: https://gist.github.com/asukakenji/f15ba7e588ac42795f421b48b... )

- no dependencies

- fast compilation

- good performance

- good debugging

- rock solid and production ready

I don't know any compiler from smaller team that matches the Go one, it's one of the reason why people use Go, compiling things is fast and easy, from my raspbery pi I can compile a win10 64bits exec with 0 problems 0 tools to install. ( ex: GOOS=windows GOARCH=amd64 go build . )

Re:Size visualization of Go executables using D3 by coldtea on Tuesday April 02, 2019 @02:32PM (#19557173)
None of those are arguments for "PL experts" as none of those are specifically signs of superior PL expertise.

Multiple platforms for example are mainly about adoption (and not having much optimizations/assembly parts in the codebase, making it easier to port).

Fast compilation is something several languages manage. And being fast to compile because you don't do much (in the way of optimizations) is something most languages can manage.

"No dependencies" is also about adoption and resources (to replace popular dependencies). Nothing particular related to PL/compiler expertise about it.

Go never had a "good debugging" story, and it doesn't have the best performance either (e.g. compare with Rust, D, Crystal, etc). In certain areas like text processing it's even worse.

Re:Size visualization of Go executables using D3 by foldr on Wednesday April 03, 2019 @12:22PM (#19564674)
> none of those are specifically signs of superior PL expertise.

If none of the things on that list is a sign of superior PL expertise, one would have to ask whether PL expertise is actually worth very much.

Re:Size visualization of Go executables using D3 by grumpydba on Tuesday April 02, 2019 @04:39PM (#19558351)
Richard Hudson, Ian Lance Taylor, David Chase, Austin Clements. The team is incredibly skilled.

And of course Pike, Griesmer & Cox.

I think that they are well aware of the trade-offs they make. And I also think that it is quite arrogant to say otherwise just because your tastes are different.

Go ahead, please write the go haters' handbook. Or write rants saying 'go is obsolete'. And the future will say who has a viable legacy.

Re:Size visualization of Go executables using D3 by ngrilly on Tuesday April 02, 2019 @03:57AM (#19551966)
> anything intended for debugging purposes only should have zero effect on the executable when not being used

It's not always possible. And debugging is very important in production (be able to inspect memory usage, generate meaningful stack trace, etc.).

Re:Size visualization of Go executables using D3 by cube2222 on Tuesday April 02, 2019 @01:39AM (#19551354)
Do you have somewhere one could read up on why those really are such antipatterns?
Re:Size visualization of Go executables using D3 by nothrabannosir on Tuesday April 02, 2019 @05:03AM (#19552254)
The same author, Raphael Poss, has another post on it:

https://science.raphael.poss.name/go-calling-convention-x86-...

It's very extensive and informative.

Re:Size visualization of Go executables using D3 by coldtea on Tuesday April 02, 2019 @11:03AM (#19554850)
>This is very surprising for a language that targets somewhat high performance.

In many cases Go has opted for ease of implementation over high performance.

Re:Size visualization of Go executables using D3 by z0r on Tuesday April 02, 2019 @12:35PM (#19555962)
Ease of compiler implementation specifically
Re:Size visualization of Go executables using D3 by senozhatsky on Tuesday April 02, 2019 @06:47AM (#19552720)
> This is very surprising for a language that targets somewhat high performance.

> Looks like it's a 5-10% performance hit, but makes it easier to provide good backtrace information

Very interesting indeed.

Well, forcing your compiler to start passing parameters via stack is pretty simple and it's not so uncommon.

And probably (and maybe) the Go team just wanted to make things simple. Otherwise they would need to have two parameter passing implementations and sometimes even use a mixed one - when platform does not have enough registers (e.g. 6 CPU registers and a function which takes 6+ params), so some parameters would be passed via registers, some via stack; or maybe all of them via stack. And so on. So I am byuing the "good backtrace" point.

For example, suppose I have a silly logging function my_func() [C code], which takes 8 params:

    __attribute__ ((noinline)) 
    void my_func(const char *module, const char *func, const char *level,
                 int line, int B, int C,
                 const char *app, const char *session)
    {
        printf("%s:%s:%s:%d %d %s %s\n", module, func, level,
               line, B + C, app, session);
    }

    int main()
    {
        my_func("core", __func__, "error", 1, 2, 3, "a.out", "dummy");
        return 0;
    }
Let's compile it with gcc (-O2) for ARM and let's take a look at what main() does:

    000103d8 <main>:
       103d8: e52de004  push {lr}  ; (str lr, [sp, #-4]!)
       103dc: e3002630  movw r2, #1584 ; 0x630
       103e0: e3402001  movt r2, #1
       103e4: e24dd014  sub sp, sp, #20
       103e8: e3003638  movw r3, #1592 ; 0x638
       103ec: e3403001  movt r3, #1
       103f0: e3001600  movw r1, #1536 ; 0x600
       103f4: e3401001  movt r1, #1
       103f8: e58d200c  str r2, [sp, #12]
       103fc: e3000628  movw r0, #1576 ; 0x628
       10400: e3400001  movt r0, #1
       10404: e58d3008  str r3, [sp, #8]
       10408: e3a02003  mov r2, #3
       1040c: e3a03002  mov r3, #2
       10410: e58d2004  str r2, [sp, #4]
       10414: e58d3000  str r3, [sp]
       10418: e3002620  movw r2, #1568 ; 0x620
       1041c: e3402001  movt r2, #1
       10420: e3a03001  mov r3, #1
       10424: eb00004c  bl 1055c <my_func>
       10428: e3a00000  mov r0, #0
       1042c: e28dd014  add sp, sp, #20
       10430: e49df004  pop {pc}  ; (ldr pc, [sp], #4)

Looks like a bunch of stores to stack ptr: + 0 bytes; + 4 bytes; + 8 bytes; + 12 bytes.

[Edit: should have passed __LINE__ instead of hardcoded 1, but that doesn't change the assembly.]

Re:Size visualization of Go executables using D3 by rurban on Tuesday April 02, 2019 @03:49PM (#19557903)
Nobody says "uses memory", you call that the stack, and that was the dominant calling convention over the last decades. It's not dead-slow, it's just not as fast as the new fastcall conventions via registers. "memory" is usually the heap, which is really dead-slow compared to the stack.

There are many VM's which do the same, they have the advantage that the GC does not need to spill all registers to the heap just to find the roots. All roots are at the stack already. Basile Starynkevitch's GC from Qish e.g. does it like this and is one of the fastest GC's around. The stack is always in the cache, and the registers can be used for more locals or intermediate results.

Re:Size visualization of Go executables using D3 by bastawhiz on Tuesday April 02, 2019 @10:24AM (#19554362)
I'm curious whether LLVM optimizes this away when using the Go front end. The last time I played with LLVM I was blown away by the types of things that it was capable of optimizing successfully.
Re:Size visualization of Go executables using D3 by amelius on Tuesday April 02, 2019 @07:06AM (#19552823)
> Go uses memory instead of registers to pass arguments and return values across function calls.

But I suppose not for inlined function calls (?)

Re:Size visualization of Go executables using D3 by kibwen on Tuesday April 02, 2019 @07:39AM (#19552993)
Last I checked (which was a very long time ago, mind), Go's inliner was very conservative, and would only inline functions that were comprised of exactly one statement.
Re:Size visualization of Go executables using D3 by weberc2 on Tuesday April 02, 2019 @08:04AM (#19553131)
This is no longer the case, but I believe it is still quite conservative.