News for hackers. Stuff that matters.
Re:On Go, Portability, and System Interfaces by smegel on Tuesday September 22, 2015 @05:55PM (#10261983)
> Basically, the Go folks want to minimize external dependencies and the web of failure that can lead to.

What is he ranting on about? The reason Go doesn't link against system libraries is that Go in binary-incompatible with C, as in it's calling convention is different. There is a way to call C through a shim but it's extremely slow which is why they implemented their own syscall interface for Go. Static linking is just a side-effect of this.

Edit: the Go team want to support dynamic linking and it is on the roadmap but not fully implemented as of 1.5 https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGh...

Re:On Go, Portability, and System Interfaces by tedunangst on Tuesday September 22, 2015 @07:11PM (#10262332)
I doubt go to C calling conventions are going to be the slowest part of calling tcsetattr(). Using ioctl() is being difficult for the sake of being difficult.
Re:On Go, Portability, and System Interfaces by pcwalton on Tuesday September 22, 2015 @07:16PM (#10262367)
> I doubt go to C calling conventions are going to be the slowest part of calling tcsetattr().

Well, by "C calling conventions" it really means "performing a stack switch to get out of M:N, allocating a big stack if necessary or taking a lock to check one out". It's quite a lot of overhead and I wouldn't be surprised if it dwarfs the cost of the SYSENTER/SYSEXIT pair in many cases.

Re:On Go, Portability, and System Interfaces by 4ad on Wednesday September 23, 2015 @06:00AM (#10264160)
It's not necessary to allocate a large stack, there is always one available because threads have their own stack, and you can switch to it at all times because goroutine threading is cooperative. In other words, if you are scheduled to run on a thread, you and only you have access to that thread stack.

Stack switching doesn't take more than a few machine instructions.

The problem with tcsetattr is that it's a library function, so you (generally[1]) need the target library in order to create a program that calls it. This is bad because in Go we value cross-compiling a lot, so we can't depend on having other people's shared libraries available in the build environment.

Syscalls, however, we know how to do them and we know what they are, so we don't need any special target library in order to generate programs for the target.

[1] On platforms like Solaris and Windows, where we are forced to use target shared libraries, we cheat a little and encode the target symbol names that we need, and we rely on standardized ELF/PE symbol resolution mechanism creating a scheme that's essentially equivalent to having import libraries available. But this is very complicated and takes a lot of code in the linker, so it's not at all unexpected that we avoid it whenever possible.

Re:On Go, Portability, and System Interfaces by pcwalton on Wednesday September 23, 2015 @12:19PM (#10265962)
> Stack switching doesn't take more than a few machine instructions.

In Rust we found that stack switching was really slow, even when fully optimized to that scheme. CPUs really don't like it. We thought "it was only a few instructions; how bad can it be"; turns out that any function call overhead can quickly add up to 2x or more overall performance penalties in some programs.

See the performance numbers in this thread for cgo (using the same technique) [1]. According to the first benchmark, calling the function via cgo is 16x slower. You're at 161ns for a no-op function call.

Consider that the cost of a syscall can be below 100ns [2]. The overhead can add up quite a bit!

[1]: https://groups.google.com/forum/#!topic/golang-nuts/RTtMsgZi...

[2]: http://forum.osdev.org/viewtopic.php?p=209429&sid=9bcec5b684...

Re:On Go, Portability, and System Interfaces by tedunangst on Wednesday September 23, 2015 @05:01PM (#10268190)
There's theory and there's practice. Saying that using ioctl is the right choice because it helps cross compilation rings a little hollow when the cross compiled binary obviously won't run due to the target system not having said syscall.
Re:On Go, Portability, and System Interfaces by 4ad on Wednesday September 23, 2015 @05:46PM (#10268476)
I'm sorry, but what?

If the system call doesn't exist, then the program won't compile, because the system call stub won't be defined for that platform.

If the system call is ioctl, and the user tries to call with an request that doesn't exist, the program won't compile because the request will not be defined.

But we don't even export ioctl! First of all, it's not type safe. We only expose ioctl wrappers that are type-safe and use requests defined by the target platform, not arbitrary integers passed by the users.

Of course if the user does an indirect system call, and mucks with unsafe, and passes it arbitrary data, than that's a user problem, not a Go problem.

Using ioctl internally by Go is the right thing.

Users indirectly calling ioctl and breaking type-safety with unsafe is just bad code, but that doesn't have anything to do with Go. Go won't prevent you if you want to shoot yourself like that.

Re:On Go, Portability, and System Interfaces by tedunangst on Wednesday September 23, 2015 @06:33PM (#10268705)
What is the linked post about if not code that could have been portable but wasn't? I'm having a hard time determining if you're saying the code in question was right or not.
Re:On Go, Portability, and System Interfaces by 4ad on Thursday September 24, 2015 @04:35AM (#10270482)
Well the code in the linked post doesn't compile on Solaris because SYS_IOCTL is not defined on Solaris. So that was my point. You don't get code that compiles but doesn't work, unless you try really hard.

But the code is bad. If you have to call system calls through the indirect system calls and use unsafe to pass parameters, you either do something wrong, or we did something wrong. In this case the author of that code is excusable. It was our fault. We didn't expose the necessary type-safe wrappers, so the authors was forced to write code like that.

This is changing however, just yesterday I committed support for termio-related stuff for Solaris. I assume Linux, BSDs and the other systems will follow very soon.

Re:On Go, Portability, and System Interfaces by tedunangst on Tuesday September 22, 2015 @07:50PM (#10262525)
All of which is likely dwarfed by all the code that runs between SYSENTER and SYSEXIT. And how many freaking times do you need to call tcsetattr()?

Even worst case, if calling overhead is a problem, the Go team can likely improve it at some point, but they are less likely to magically map special cases of syscall.SYS_IOCTL to tcsetattr for the sake of portability.

Re:On Go, Portability, and System Interfaces by 4ad on Tuesday September 22, 2015 @08:16PM (#10262608)
Not sure I understand your point, tcsetattr calls a system call on its own...

So the alternatives are between doing a trivial system call on the Go stack, or doing a complex stack switch operation that does the (too complicated) System V calling convention ABI translation, and then calls a function that does a system call...

Re:On Go, Portability, and System Interfaces by nulltype on Tuesday September 22, 2015 @08:11PM (#10262594)
If you avoid calling c code, you don't have to use cgo, which may be a better reason than "being difficult".
Re:On Go, Portability, and System Interfaces by 4ad on Tuesday September 22, 2015 @08:19PM (#10262617)
> The reason Go doesn't link against system libraries is that Go in binary-incompatible with C, as in it's calling convention is different.

I'm sorry, but the system call "calling convention" is different too. This argument doesn't hold water. Calling convention translation is a trivial issue. Go doesn't usually use system libraries in order to minimise dependencies and to allow easy cross-compiling.

> There is a way to call C through a shim but it's extremely slow

Not really, it's actually pretty fast. Of course it will be always slower than just a regular function call, but it's sufficiently low-overhead that in most cases you can ignore the overhead. Of course there are still cases where you can't ignore the overhead.

On systems like Windows and Solaris, the cgo mechanism is actually used to do back-end "system calls" all the time.

Re:On Go, Portability, and System Interfaces by smegel on Tuesday September 22, 2015 @10:09PM (#10262975)
> I'm sorry, but the system call "calling convention" is different too.

Sufficiently different to not be a problem? As I understand it system calls don't usually rely on the stack whereas C calls do, and it is the stack mismatch between C and Go that is the issue.

> but it's sufficiently low-overhead that in most cases you can ignore the overhead

Maybe for user called function...but if every single system call had to swap out the stack surely that would have a pretty heavy impact...

Re:On Go, Portability, and System Interfaces by geofft on Tuesday September 22, 2015 @10:54PM (#10263085)
The Go calling convention passes everything, in both directions, on the stack. Most platform-native ("C") calling conventions put as many things in registers as possible, and reuse one of the registers for the return value when possible. However, this is just a convention from one function to the next, and they can share the same stack; the caller of a C standard library function merely needs to know what to put on the stack and what to put in registers (and which registers to save). And in fact, the C calling convention is more efficient than Go's from the caller's point of view (fewer things have to go on the stack, some registers are promised not to be clobbered), so calling a C library function with the native calling convention is, on all reasonable platforms, a fine idea performance-wise.

There is the problem of the stack itself. The C model envisions a large, contiguous stack, possibly with some kernel magic to grow the stack as you approach the end (by handling page faults). Go previously used "segmented stacks", essentially a linked list of heap-allocated buffers used as stack, where functions knew how to allocate and deallocate segments when necessary. C doesn't know how to do that, so a C call previously would have to allocate plenty of memory just in case the function wanted to use a bunch of stack space.

Go no longer uses segmented stacks, so that's essentially not a concern. Go still does reallocate stacks, but does so by copying the entire stack, and the stack is basically never freed. So there might be a one-time performance penalty on the first C call (unless the compiler is smart enough to tell when a Goroutine might possibly call into C, and pre-allocate that space), or the first C call after most of the existing stack was used in Go code, but not after that.

tl;dr: The C calling convention is strictly faster than the Go one (but more complicated), and in Go 1.3+ the existing stack is fine to use, so there shouldn't be performance concerns.

(Unless, of course, you think the C library itself is slow. But that's a very different problem.)

Re:On Go, Portability, and System Interfaces by 4ad on Wednesday September 23, 2015 @05:46AM (#10264125)
> and the stack is basically never freed

Not true, stacks are still freed if utilisation drops below a certain percentage.

> So there might be a one-time performance penalty on the first C call [...] in Go 1.3+ the existing stack is fine to use

Not true, contiguous stacks start even smaller than segmented stacks, also they can't be used for C code at all. Only type-safe code with stack maps can run on the goroutine stacks, because the precise garbage collector has to scan the stack and has to understand where on the stack there are pointers and where there aren't. C code breaks this requirement.

For calls into C, the thread stack is used, which in Go usually has 32kB, but varies on some platforms, for example on Windows, it's 256kB IIRC. The runtime switches the stack (which only takes a few instructions), does the ABI translation and calls the function.

The stack switching mechanism and ABI translation mechanism have trivial overhead, most of the overhead comes from runtime bookkeeping, e.g. the runtime must ensure there are always enough threads to run user-level Go code.

Re:On Go, Portability, and System Interfaces by geofft on Thursday September 24, 2015 @10:15PM (#10276060)
> also they can't be used for C code at all. Only type-safe code with stack maps can run on the goroutine stacks, because the precise garbage collector has to scan the stack and has to understand where on the stack there are pointers and where there aren't. C code breaks this requirement.

Is this still true of C code that doesn't call back into Go code, like libc itself? The Go garbage collector will never get triggered, and any Go code should treat anything below %esp after the call completes as uninitialized data.

(Also, on Windows, don't actual system calls reuse the same stack, too?)

It seems like if you wanted to design Go for dynamically linking to libc instead of making raw system calls, it wouldn't be difficult.

Re:On Go, Portability, and System Interfaces by 4ad on Friday September 25, 2015 @07:35AM (#10277529)
> Is this still true of C code that doesn't call back into Go code, like libc itself? The Go garbage collector will never get triggered, and any Go code should treat anything below %esp after the call completes as uninitialized data.

I should give a more thorough explanation.

If you call C code, the only way that would be useful is if we pass Go-allocated buffers to the C code. This only works if we pass to the C code untyped uintptr's, which must live on the stack because of garbage collector reasons.

In your hypothetical world, you want to call C code on the g stacks. This can only works if the g stacks are copied into a larger stack. The stack copying mechanism inspects the stack map to see which pointers need updating. This stack map is also used by the garbage collector, but I shouldn't have mentioned it in my original post.

The stack copying mechanism can't update the untyped uintptr's living on the stack, therefore you can't call C code on the g stack.

There are other reasons why calling C code on the g stack would be a bad idea but I won't get into them. Final point is, C code is called on the system stack.

> (Also, on Windows, don't actual system calls reuse the same stack, too?)

Which stack and which system calls?

On Windows, target calls are made on the g0 stack, which is the thread stack. These calls are just calls into regular C code, most of it in kernel32.dll, some of it in ntdll.dll. This C code will eventually do system calls. Those actual system calls don't need any stack.

Re:On Go, Portability, and System Interfaces by 4ad on Wednesday September 23, 2015 @05:39AM (#10264098)
> Sufficiently different to not be a problem? As I understand it system calls don't usually rely on the stack whereas C calls do, and it is the stack mismatch between C and Go that is the issue.

Switching the stack only takes a few machine instructions. Most of the overhead comes from runtime bookkeeping, in contrast ABI translation and switching the stack take a trivial amount of code and time.

> Maybe for user called function...but if every single system call had to swap out the stack surely that would have a pretty heavy impact...

As mentioned, on some platforms, for example Solaris and Windows, this always happens, and even on other platforms, in some conditions, it happens too. Not to mention that the runtime switches stacks for its own reasons all the time. The overhead is totally dwarfed by the cost of doing system calls in the first place.

Re:On Go, Portability, and System Interfaces by readams on Tuesday September 22, 2015 @06:07PM (#10262035)
Static linking is independent of this concern. Go could implement dynamic linking but doesn't for ideological reasons.

I'd really hate to be a distribution maintainer in a world where go components are common. Security bug in the TLS library? Time to download 14GB of updates!

Re:On Go, Portability, and System Interfaces by xyproto on Tuesday September 22, 2015 @06:13PM (#10262075)
Not ideological reasons. The latest version of Go supports dynamic linking for some platforms.
Re:On Go, Portability, and System Interfaces by pjmlp on Tuesday September 22, 2015 @06:18PM (#10262099)
On Android there is no way around it, as native code can only be accessed via dynamic linking.

This is what some Go designers think of dynamic linking:

http://harmful.cat-v.org/software/dynamic-linking/

Re:On Go, Portability, and System Interfaces by GauntletWizard on Tuesday September 22, 2015 @10:39PM (#10263048)
And those reasons are clear and clear-cut. Dynamic linking is a net negative in most use cases, and certainly in all the use-cases that the Go developers are developing for, where reproducible builds are mandatory, code size is a tiny fraction of main memory, and code quality is very strictly maintained. When you've built a good development and deploy strategy, rebuilding and redistributing your binary is an everyday activity. There's no additional cost.
Re:On Go, Portability, and System Interfaces by shadowmint on Tuesday September 22, 2015 @09:15PM (#10262806)
Yeeesss... but ultimately, Go is also a pragmatic language.

Dynamic linking is coming to Go, like it or not; because it's a feature that is needed in certain circumstances, and that has been acknowledged, even by the Go team.

Re:On Go, Portability, and System Interfaces by bahamat on Wednesday September 23, 2015 @01:04AM (#10263404)
That same web page lists Linux as harmful.

http://harmful.cat-v.org/software/operating-systems/linux/

Re:On Go, Portability, and System Interfaces by marssaxman on Tuesday September 22, 2015 @06:22PM (#10262109)
I'm happy to be an end user in a world where go components are becoming more common, because statically-linked executables are far less likely to break when I upgrade unrelated programs.
Re:On Go, Portability, and System Interfaces by nulltype on Tuesday September 22, 2015 @06:11PM (#10262060)
Just download the deltas instead?