News for hackers. Stuff that matters.
Re:Why are my Go executable files so large? by swiftcoder on Sunday December 29, 2019 @01:21PM (#21907431)
This is where letting a large enterprise guide the development of a piece of widely-used software becomes questionable. At a FAANG the constraints are fundamentally different.

At work I routinely see CLIs programs clocking in at a gigabyte, because it was simpler to statically link the entire shared dependency tree than to figure out the actual set of dependencies, and once your binary grows that big, running LTO adds too much time to one's builds. And disk space is basically free at FAANG scale...

Re:Why are my Go executable files so large? by jrockway on Sunday December 29, 2019 @10:28PM (#21910586)
> This is where letting a large enterprise guide the development of a piece of widely-used software becomes questionable. At a FAANG the constraints are fundamentally different.

I see where you're going with this, but the conclusion of the article is that Go isn't (what you call) FAANG-y enough for them.

From the article: "This design choice was intended to lower the start-up time of programs [...] This performance goal is not relevant to server software with long-running processes, like CockroachDB, and its incurred space cost is particularly inconvenient for large, feature-rich programs."

What they want is a way to pass 1000 parameters to the compiler so that nobody ever gets a working binary unless they have a team of release engineers to make one for you. The Go team and their users at Google easily have access to said team of release engineers, but took the opposite approach. They optimized for programs that start fast even though most of those teams at Google are writing programs that run for a long time. They optimized for a compiler and runtime that don't have a billion knobs to turn, even though they can afford to pay an army of engineers to tune those knobs.

So I actually think this has very little to do with optimizing for FAANG. This is more about every program getting the same compiler options, which is merely a philosophy that some people on the Go team have, not really a philosophy that every FAANG has. (They use Java at Google too, and Java is more than happy to give you knobs to turn. It also doesn't start up very fast.)

Re:Why are my Go executable files so large? by thu2111 on Monday December 30, 2019 @04:03AM (#21911925)
Maybe with bad C toolchains you can get combinations of switches that result in dead binaries, but I've never seen a JVM that wouldn't start because of bad tuning flags. At most you can hurt (or help) performance.

I'm not sure the Go 'philosophy' here has anything to do with it. If you click through some links you'll end up at a GitHub issue where they discuss a prototype for making Go's code generation not completely awful. The justification for not pursuing a 5-10% improvement (which is huge for compiler opts) is that it'd break existing inline assembly by changing the calling convention. This isn't a philosophical objection, it's just bad design painting them into a corner.

Re:Why are my Go executable files so large? by pjmlp on Monday December 30, 2019 @03:56AM (#21911879)
Java starts up pretty fast if you bother to AOT compile it, plenty of commercial options available since early 2000's.

For the free beer folks there is AOT support on OpenJDK, OpenJ9 and GraalVM.

Re:Why are my Go executable files so large? by BossingAround on Monday December 30, 2019 @02:42AM (#21911532)
> It also doesn't start up very fast

This has been changing rapidly with GraalVM and projects like Quarkus, which can build a static binary of a Java program that starts, as you'd expect, much faster (still not performance of a C/C++ program, but way faster than a regular Java program).

Re:Why are my Go executable files so large? by jhgb on Tuesday December 31, 2019 @04:21AM (#21919882)
Did I miss something important about GraalVM? I thought it was another runtime on top of JVM, not a replacement for it. For a Java program, for example, GraalVM would presumably have no impact since it wouldn't be used for it.
Re:Why are my Go executable files so large? by thu2111 on Friday January 03, 2020 @06:20AM (#21944963)
GraalVM is several things.

In this context people are talking about the native-image tool which uses a small JVM written in Java (SubstrateVM) to compile to native code ahead of time, no HotSpot or other JVM dependencies.

There's also Truffle which is a way to put other languages onto the JVM (both HotSpot and SubstrateVM).

Re:Why are my Go executable files so large? by tomphoolery on Sunday December 29, 2019 @01:35PM (#21907536)
Disk space in general is pretty much free these days. 123mb for a whole database is really not that big of a deal, IMHO. For example, my local PostgreSQL docker image is 140mb plus the alpine OS (5mb). And the Ruby on Rails application using that image clocks in at a little over 1gb, also using the ruby alpine image as a base (50mb).

With my company, the cost really started to become a burden with data transfer. But transferring images to and from the AWS container registry is so expensive that we actually build production images inside the Kubernetes cluster (plus the cluster has access to all the secrets and stuff), even though it was a bit harder to implement.

If you're FAANG and you can run your own stable and highly-available cloud, data transfer rates don't matter, so you can deploy your application the "right" way in a containerized world.

Re:Why are my Go executable files so large? by gumby on Sunday December 29, 2019 @01:52PM (#21907647)
Yes but when running a long-lived application on a lot of data it's typically important to keep the executable small both so it can be "hot" and to leave more room for data. At scale this could be even more important, not less, than for a smaller operation.

Of course the real (i.e. explicitly stated by Pike) driver for go was the assertion that inexperienced new hires write poor code and so the harder it is for them to get into trouble the better, even at the cost of other issues.

Re:Why are my Go executable files so large? by boardwaalk on Sunday December 29, 2019 @02:25PM (#21907872)
Executable size on disk doesn't dictate effective size at higher levels of the memory hierarchy.

IOW, you're paying by the page/cache line. If the extra bloat (debug information) in your executable isn't interspersed within your actual code (it shouldn't be), you aren't paying for it in runtime efficiency.

Re:Why are my Go executable files so large? by lazyjones on Sunday December 29, 2019 @02:28PM (#21907884)
> Yes but when running a long-lived application on a lot of data it's typically important to keep the executable small both so it can be "hot" and to leave more room for data.

Who really runs server applications these days where data/rss are not a large multiple of the actual code segments? People happily run JVM server processes these days, how much code/data do you think that pulls in just to start up?

Re:Why are my Go executable files so large? by akhilcacharya on Sunday December 29, 2019 @03:19PM (#21908169)
Genuinely wondering how a CLI would be 1GB, even at Facebook. Never encountered anything _close_ at Amazon.
Re:Why are my Go executable files so large? by lkramer on Sunday December 29, 2019 @05:02PM (#21908867)
Not your usually usecase, but the binary for running the Bloomberg terminal is well over 1GB (In fact running into the the 4GB executable boundary was a problem).
Re:Why are my Go executable files so large? by akhilcacharya on Sunday December 29, 2019 @05:32PM (#21909061)
Oh, that's fascinating. I never checked the binary size but that makes sense.
Re:Why are my Go executable files so large? by saagarjha on Monday December 30, 2019 @02:46AM (#21911553)
I’m curious, what’s in it? Which sections take up the space?
Re:Why are my Go executable files so large? by lkramer on Monday December 30, 2019 @03:08AM (#21911632)
Sorry, I should have clarified, this is the server side binary. I believe the part that's run on the user's computer is just a kind of dumb terminal.

The reason it's so big (or at least was several years ago) is that every single function is compiled into one binary monolith.

Re:Why are my Go executable files so large? by ComputerGuru on Monday December 30, 2019 @03:57AM (#21911888)
I’m sorry, that doesn’t sound possible. A clean Windows 10 install (kernel plus userland) can clock in at 6GB and you’re telling me the Bloomberg server CLI was hitting the 4GB wall just because it’s statically linked? No amount of pure code gets that big, it must be something else in the tooling or object format. Perhaps it includes binary assets.
Re:Why are my Go executable files so large? by thu2111 on Monday December 30, 2019 @04:07AM (#21911953)
That's quite believable to me for an old codebase. Command line utilities at Google routinely reached hundreds of megabytes in binary size due to static linking of everything. A Bloomberg server is surely far larger and more complex than those tools, so gigabytes isn't unbelievable.

Binary code can be quite bloated compared to source code. It depends a lot on the encoding. The same author as this blog post has another actually much more interesting one on how Go compiles common constructs. It's rather enlightening as the example generated assembly (implicitly) explains why Go compiles so fast. It has to be the dumbest, least optimising compiler I've ever seen. Very short and trivial source code constructs like function calls balloon into pages of instructions, far far more than a C compiler would generate.

C++ compilers also have a reputation for generating bloaty code, but that's mostly due to compile-time specialisation of containers and the way template expansion works. If you write C-as-C++ then you get pretty tight binaries.

Re:Why are my Go executable files so large? by lkramer on Thursday January 02, 2020 @10:14AM (#21935969)
I think you're underestimating just how much the bloomberg terminal backend does. It's 40 years of obscure functions packed in to one massive binary.
Re:Why are my Go executable files so large? by swiftcoder on Sunday December 29, 2019 @04:39PM (#21908695)
I never saw one a singular tool that big at Amazon either, but the general concept held. The AWS service I worked on had ~1,000 jars in its dependency closure. All of those had to be uploaded to S3 after each build, and then downloaded onto each EC2 instance during deployment.

We're talking on the order of a terabyte of data transfer each time we deployed to a thousand instance fleet (ideally deploying weekly)

Re:Why are my Go executable files so large? by bpodgursky on Sunday December 29, 2019 @01:36PM (#21907540)
Mmmm I don't know if I agree with that logic.

I'd argue the opposite -- as a startup, you can't afford to micro-optimize. Labor, time and opportunity cost dwarf all but the grossest resource waste. If you need to use 100GB/k8s node instead of 50GB, it will have 0 effect on the success of your venture.

At Google scale, it becomes worth it to optimize:

- You are delivering more product per engineer, and more product means more resources. Instead of a single customer instance which costs $1/month more, you have 100,000 customer instances, costing a significant amount more. It becomes worth trimming margins.

- You have economies of scale, and it might be worth it for an engineer to spend a month trimming 2% of the cost of a software deliverable.

The common refrain for startups is "do things that don't scale", and this is for good reason. Google has to actually worry about fixing things AFTER they are scaled.

Re:Why are my Go executable files so large? by fwip on Sunday December 29, 2019 @02:15PM (#21907806)
I think you're both right.

A 125MB binary is relatively beefy, but still easily fits in RAM. The amount of disk that you're spending on a single executable (your database, in this instance) is tiny in comparison to the amount of data stored in that database.

It's definitely worth it for Google to trim 2% off of their storage requirements - but if your binary is 0.1% of your storage, it's barely even worth glancing at.

Re:Why are my Go executable files so large? by TheReveller on Sunday December 29, 2019 @03:41PM (#21908315)
There are plenty of efforts and tooling that reduce the number of dependencies and the deploy size of binaries at Google. The notion that we don't care about size isn't true.

But it's true that it's not worth optimising first, it's done by first evaluating the impact across the fleet and then prioritising the most effective changes.

Re:Why are my Go executable files so large? by BubRoss on Sunday December 29, 2019 @02:03PM (#21907718)
LTO is no magic bullet for binary size either. A binary that does nothing will still link in the whole C library. It doesn't end up decreasing large program sizes that much in my experience either.
Re:Why are my Go executable files so large? by techsin101 on Sunday December 29, 2019 @02:47PM (#21908003)
Tree shaking?