I love Rust, but Go is focused on a handful of very specific use cases. Rust is not. I don't know that I can fault Go for choosing implementation details that directly enable those use cases.
[1]: http://dtrace.org/blogs/wesolows/2014/12/29/golang-is-trash/
Specifically the top reply there is by rsc (tech lead for Go)
But wouldn't it still be nice to have a standardized way to analyze post-mortem dumps across languages?
Unfortunately with Go I become less convinced with every passing year that they can keep getting away with this. They keep spinning obvious weaknesses as philosophical strengths, rather than admitting it is due to limited resources and backwards compatibility constraints. Their use cases (servers, UNIX tools) aren't actually unusual or different to other teams. It seems like every time I read about Go they've made what is simply a bad design decision that they later regret and explore fixing, but their rules cause them to keep compounding self-inflicted wounds. Compared to other language and runtime teams they just don't seem to know what they're doing.
Here are just some of the examples we've learned about so far.
Stack unwinding and hyper-inefficient calling conventions. Despite being designed for servers at Google, where throughput really matters a lot, they generate extremely bloated code and hardly use registers rather than generating unwind metadata that's consulted on need and using a tuned calling convention. Tuned calling conventions are optimisations that date back literally decades in the C world and yet Go doesn't have them!
This significantly reduces icache utilisation (hurting throughput) and means they can't use any existing tools, and yet the only benefit is it made their compiler easier to write initially. They increased the server costs of Go shops permanently by taking this shortcut which benefited only the compiler authors. Now they struggle to fix it because they don't have any de-optimisation engine either, so changing calling conventions makes it harder to get useful stack traces and would break user-authored inline assembly (which is rare for Go's use cases).
Compare to how the Java guys did it: the compiler generates highly optimised code and tables of metadata that let the runtime map register/stack state back to the programmers de-optimised view of the program. Methods can be inlined aggressively because the VM can always undo it, so it doesn't get in developer's way. That metadata is only consulted when a stack unwind is actually needed, which is rare. The rest of the time it sits cold in far away RAM or swapped to disk. Calling conventions aren't exposed to the user and can be changed as needed, but if you need custom assembly you go via JNI that uses the platform calling convention and accept a slower function call.
Go's approach isn't some principled matter of design, as evidenced by their explorations of fixing it. They just didn't plan ahead.
Garbage collectors. The Go team originally tried to claim their GC was some sort of massive advance, a GC for the ages. A few years later they gave a presentation where they admitted they had explored replacing it several times because it's extremely inefficient, but are hamstrung by a (self imposed) rule that they're only allowed one knob and don't want to make their compiler slower. Once again, the constraints of their compiler causes massive cost bloat for projects in production (where cost really matters).
Compare to Java: the default GC tries to strike a balance between throughput and latency, but if you need super low latency or super high throughput you can flip a switch to get that. The runtime can't know if your task is a batch job like a compiler or a latency sensitive HTTP server, so can tell it, but if you don't it'll take a middle path. Given the huge costs of large server farms, this is sensible!
Compile time. Whenever you read about the Go team's choices it's apparent they are willing to mangle basically anything to get themselves an easier to write or faster compiler, hence the fact that it hardly optimises and generates massive binaries. But this isn't the only way to get fast compile times.
Compare to Java: compilation is done in parallel with program execution and only where it matters. During development where you frequently start up and shut down programs, you're only waiting for the compiler frontend (javac) which is very simple and doesn't optimise at all, so it's fast like Go's is. When deployed to production the program automatically ends up optimised and running at peak performance and you don't even need to flip a "in prod" switch like with a C compiler: the fact that the program is long running is itself evidence that you're in prod and worth optimising.
This heuristic used to hurt a lot for small command line tools, which usually don't need to be very fast. But you can produce binaries with the GraalVM native-image tool that start as fast (or even faster) than C programs do now, so that's not a big deal any longer.
Generics. Well, this one has been thrashed out so much I won't cover it again here. Suffice it to say that other languages have all concluded this is worth having and managed to introduce it in either their first versions, or in a backwards compatible way later.
Debugging. The ability to easily debug binaries and get reasonable stack traces is known to make program optimisation hard, because optimisation means re-arranging the program behind the developer's back. It's hard to put a breakpoint on a function that was deleted by the optimiser, or inlined. That's why C compilers have debug vs non-debug modes. Debug binaries can be significantly slower than release binaries, hence the difference. In fact in the past I've seen cases where debug-mode C binaries were so slow you couldn't use them because getting the program to the point where it'd experience issues took so long. And of course forget about debugging production binaries.
Golang faces the same problem but in effect just always runs every program in debug mode.
Compare to Java: See the above description of the de-optimisation engine and tables. If you request a stack trace or probe a method with a method, the program is selectively de-optimised so the part being inspected by the developer looks normal whilst the rest of the program continues running at full speed. This means you can attach debuggers to any program at any time, without flags, and you can even attach debuggers to production JVMs at any time. This feature doesn't impose any throughput hit (it does consume memory, but it's cold memory).
So we can see that repeatedly the Go guys have made choices that seem to have wildly wrong cost/benefit tradeoffs, tradeoffs that literally nobody else made, and almost always the root cause is their duplication of effort vs other open source runtimes. They use a variety of fairly condescending justifications for this like "our employees are too young to handle more than one flag", but when you dig in you find they've usually explored changing things anyway. They just didn't succeed.
> Instead of creating (or borrowing from Plan9) an “assembly language” with its own assembler, “C” compiler (but it’s not really C), and an entire “linker” (that’s not really a linker nor a link-editor but does a bunch of other stuff), it would have been much better to simply reuse what already exists.
"Simply reuse what already exists"...like the things that they reused, for example?
IIRC Go started out shipping with a port of the Plan9 C compiler and toolchain - it was bootstrapped by building the C compiler with your system C compiler, then building the Go compiler. Which, until re-written in Go circa-2013, was in Plan-9 style C. It all looks deeply idiosyncratic but it was a toolchain the initial implementors were highly familiar with.
Perhaps their IP requirement would not sufice Google lawyers?
Perhaps Go devs would rather have more control on the development of assembler by writing it from scratch to understand every design decision instead of inheriting thousands of unknown design decisions?
I don't know. Neither do others outside of the project.
I find these baseless micro-aggressions against Go missplaced and unfruitful.
> I find these baseless micro-aggressions against Go missplaced and unfruitful.
Hu? Ok then Go is perfect because it is developed in secret.
We are discussing here, I'm not "micro-agressing" anyone. If I don't like a design / re-implementation decision, and I in the mood to share that opinion with this cyber-assembly, I do it. And I expect developers to not be offended by me having a technical opinion; and I expect third parties to be even less offended. And yes, it might be a bad opinion in some cases. I'm not even 100% sure it is not the case here, because like you said they could have had some kind of justification to do that. But it suspect it is extremely rare to have a good justification to rewrite an assembler, with really big quirks on top of that, when they did it.
Didn't implied that at all. No need for straw man.
> You can only justify so many reinventions of the wheel, yet they redid everything.
Don't extrapolate what I write.
We're adults here, can we please not talk like tumblr blog posters?
The days when the Go project fired up were different than the days when Rust started. Rust made different tradeoffs by relying on LLVM and it has advantages (free optimizations!) and disadvantages of their own.
This is a large part of why Go is not based on GCC or LLVM, why it has its own linker, its own assembly language, its own syscall interface, its own debug format, its own runtime (forgoing libc), and so on. Clearly Go's designers were more than a little contrarian in their way of doing things, but that's not the whole answer.
Being able to repurpose existing code is an efficiency multiplier during the bootstrapping phase. But when bootstrapping is done, you have to consider the ROI of going back and redoing some things or keep a design that works pretty well. The Go team is undoubtedly aware of some of these issues, but probably don't consider them to be a priority.
In some cases the tools are a benefit. Go's compiler and linker are extremely fast, which I appreciate as a developer. A possible compromise would be to offer a slower build pipeline for production builds, which made use of LLVM and its many man-years of code optimizations.
I think starting with LLVM was the right decision (and one that was I was primarily responsible for). Rust would lose most of its benefits if it didn't produce code with performance on par with C++. LLVM is not the fastest compiler in the world (though it's not like it's horribly slow either), but its optimization pipeline is unmatched. I don't see replicating LLVM's code quality as feasible without a large team and a decade of work. Middling code gen performance is an acceptable price to pay until we get Cranelift; the alternative, developing our own backend, would mean not being able to deploy Rust code at all in many scenarios.
https://www.phoronix.com/scan.php?page=article&item=gcc-clan...
Of course, benchmarks, yada yada, but at least is some sort of comparison axis where the improvement over the years is clear.
Typically, developers can afford to throw more cores and more ram at their build machines.
The git support was added to make it a bit easier outside plan9.
Almost everyone else doesn't have a stable ABI below the (C) linker level.
https://www.kernel.org/doc/Documentation/ABI/README
Most interfaces (like syscalls) are expected to never change and always be available.
https://www.kernel.org/doc/Documentation/ABI/stable/syscalls This interface matches much of the POSIX interface and is based
on it and other Unix based interfaces. It will only be added to
over time, and not have things removed from it.
This is literally an explicit promise of the Linux kernel; distros have no influence over the Linux syscall ABI whatsoever.I think you're perhaps extremely confused about the difference between the userspace syscall -> kernel interface, and kernelspace API/ABI such as out-of-tree kernel modules might use. About the latter, yes, there are no API/ABI guarantees in vanilla Linux.
That's an even better description of Windows. The macOS system call table isn't officially stable, but it's at least slow to change. The Windows equivalent has been known to change from service pack to service pack.
That being said, yes, we see integration into the parent platform as being an important design constraint for Rust. I think Go made reasonable choices for what they're trying to do, though. It's all tradeoffs.
It makes the Go concurrency mechanism possible, this is not just a kind of whim.
Most importantly, this allows the scheduler to hook on syscalls in order to schedule an other routine. But this also allows to control what happens during a syscall, since the libc tends to do more than just calling the kernel in its syscall wrappers, which might not be thread safe or might not play well with stack manipulations.
This has never been a problem in my experience.
It opens you up to a bit more of behavior changing in the future, but just a tiny bit more. No need to make a big deal out of it. It's a very normal thing in software. Nobody is going to promise you a perfect stable interface to rely on forever, not even Linux. But syscalls are actually pretty easy to keep up with, they change slowly, and it's easy to detect kernel version and choose appropriate wrappers to use with very little extra code.
OS X problem is its own thing. Apple making breaking changes is not a new thing. I use an Apple laptop super rarely and still got fed up with breaking changes, even not upgrading past 13.6 at the moment.
Linux maintains stable syscall interfaces. BSDs don't guarantee it, but generally don't change much. But macOS and Windows can and will change their interfaces.
But right and wrong aside, there's the practical matter of reality. You can't simply pretend everything works how you want them to. At the end of the day you have to deal with how they actually work.
The Linux project needs to be able to evolve independently of other projects, so they do just that.
This is a classic case of technical architectures following most of the time social structures.
> Systems calls form an API and it needs to be stable and managed.
In some cases (e.g; nearly all the other oses...) system calls form an internal API, and they don't need to be stable, and they actually even don't need to be accessible except to intermediate layers provided in a coordinated way.
My preference is that the Windows/OS X model is where the boundary should belong.
glibc doesn't really "want" to be statically linked. It can be done sometimes, depending on how it's used, the phase of the moon and so on, but breaks from time to time until it's repaired.
Some of the issues are fundamental. For instance the dynamic linker is a part of the C libraries. If you statically link libc and then dynamically load another shared library, you can end up with two copies of the C libraries loaded at once.
The C library expects various external files to be found on disk, in a format that isn't totally forwards/backwards compatible. That's reasonable when things are dynamically linked because the linker and C library abstract the developer from format changes. If you statically link you should really statically link the data files too, but the C toolchain has no provision for that.
Based on what data?
At least that's the common complaint for Alpine docker images... It's based on musl and halve of the community always complains about serious performance degradation
though even java is faster in most benchmarks
So how much they got right regarding Docker's success and Go is a bit debatable.
Perhaps you are thinking of a different project?
https://archive.fosdem.org/2019/schedule/event/kubernetesclu...
The original prototype that saw a public repo was written by Joe and Craig in Python (mostly Craig IIRC) and lasted for all of about a week before they switched to Go.
The original crew of contributors, all from Google, came from a Java background and definitely wrote Java-flavored Go, but no version of Kubernetes was ever written in Java.
Source: I know all the principals and was a contributing member when Kubernetes was initially released.
That said, I agree that Docker was the first major project written in Go many people were exposed to and probably had some influence.
It is relevant in the sense that Docker isn't 100% Go nowadays.
I found these tickets a few weeks ago and they explained why the Go developers haven't yet made this change.
The goal is not to GC stack frames. But I'm not sure why the didn't create an inner class to hold the closed-over variables in non-final fields (moving them from the stack to the heap) for both the function and all closures it creates.
(Obligatory "doctor, it hurts when I use mutable state!")
Nitpick, but saying copies in Java can get confusing. Both primitives and references are bound by value. I'm sure you know, but for others: no objects are copied.
I always found this limitation had reassuring regularity; it's the same way arguments are bound to function parameters (minus bring final). Local variables being isolated from "other scopes" means that any interthread communication must be mediated through objects.
Android Java is the one making use of anonymous inner classes instead.
[0]: https://github.com/frohoff/jdk8u-jdk/blob/master/src/share/c...
Not really, because the class file with invokedynamic bytecodes is supposed to work across all JVM implementations.
Hotspot's just happens to spin an inner class at runtime.
I believe this is it.
Edit: it seems that store-load forwarding is actually slightly slower than L1: https://www.agner.org/optimize/blog/read.php?i=854#854
For context, gcc was first released in 1987 i.e. about the same time that Plan 9 started.
Go authors didn't attempt to re-invent asm syntax. They re-used the work they did over 30 years ago.
And at the time Plan 9 happened it was hardly re-inventing anything either. It was still the time of invention.
References:
Intel's Asm syntax was defined in 1978 with the release of the 8086, and the 32-bit superset in 1985 with the 386. CP/M, DOS, and later Windows assemblers all used the official syntax.
It's all just tradeoffs in the end - I think golang is finding some success because they didn't make the same tradeoffs everyone else did.
Welp, so much for Go.
Basically, while (e.g.) Python's mandated formatting style arose from Guido van Rossum's philosophy of best programming practice, Go's mandated style arose from the fact that it was easier to implement from the point of view of compiler authors who had evidently never used a lexer before, much less written one.
[1] https://stackoverflow.com/questions/17153838/why-does-golang...
can you expand, because I can't see their dysfunctional decision making from the link you provided.
"Go uses brace brackets for statement grouping, a syntax familiar to programmers who have worked with any language in the C family. Semicolons, however, are for parsers, not for people, and we wanted to eliminate them as much as possible. To achieve this goal, Go borrows a trick from BCPL: the semicolons that separate statements are in the formal grammar but are injected automatically, without lookahead, by the lexer at the end of any line that could be the end of a statement. This works very well in practice but has the effect that it forces a brace style."
Nothing about that is OK. Nothing about it makes sense.
Sure, the existing solutions might not be perfect, but reinventing the wheel gets tiresome after a while. Contrast this with Rust, which has made an overt effort to fit into existing tooling: symbols are mangled using the C++ mangler so that gdb and friends understand them, rust outputs nice normal DWARF stuff on Linux so gdb debugging just works, Rust uses platform calling convention as much as possible, etc. It means that a wealth of existing tooling just works.