I would argue it's not (very) hard data that we need in this case. My opinion is that the resource usage of infrastructure code should be as low as possible so that most resources are available to run applications.
The economic viability of application development is very much determined by developer productivity. Many applications aren't even run that often if you think of in-house business software for instance. So application development is where we have to spend our resource budget.
Systems/infrastructure code on the other hand is subject to very different economics. It runs all the time. The ratio of development time to runtime is incredibly small. We should optimise the heck out of infrastructure code to drive down resource usage whenever possible.
GC has significant memory and CPU overhead. I don't want to spend double digit resource percentages on GC for software that could be written differently without being uneconomical.
I will assume by "infrastructure code" you mean things like kernels and network stacks.
Unfortunately there are several intertwined issues here.
First, we pay in completely different ways for writing this software in manually-managed languages. Security vulnerabilities. Bugs. Development time. Slow evolution. I don't agree with the tradeoffs we have made. This software is important and needs to be memory-safe. Maybe Rust will deliver, who knows. But we currently have a lot of latent memory management bugs here that have consistently clocked in 2/3 to 3/4 of critical CVEs over several decades. That's a real problem. We aren't getting this right.
Second, infrastructure code does not consume a lot of memory. Infrastructure code mostly manages memory and buffers. The actual heap footprint of the Linux kernel is pretty small; it mostly indexes and manages memory, buffers, devices, packets, etc. That is where optimization should go; manage the most resources with the lowest overhead just in terms of data structures.
> GC has significant memory and CPU overhead. I don't want to spend double digit resource percentages on GC for software that could be written differently without being uneconomical.
Let's posit 20% CPU for all the things that a GC does. And let's posit 2X for enough heap room to keep the GC running concurrently well enough that it doesn't incur a lot of mutator pauses.
If all that infrastructure is taking 10% of CPU and 10% of memory, we are talking adding 2% CPU and 10% memory.
ABSOLUTE BARGAIN in my book!
The funny thing is, that people made these same arguments back in the heyday of Moore's law when we were getting 2X CPU performance ever 18 months. 2% CPU back then was a matter of weeks of Moore's law. Now? Maybe a couple of months. We consistently choose to spend the performance dividends of hardware performance improvements on...more performance? And nothing on safety or programmability? I seriously think we chose poorly here due to some significant confusion in priorities and real costs.
That, and things like database systems, libraries that are used in a lot of other software or language runtimes for higher level languages.
>The actual heap footprint of the Linux kernel is pretty small
And what would that footprint be if the kernel was written in Java or Go? What would the performance of all those device drivers be?
You can of course write memory efficient code in GC languages by manually managing a bunch of buffers. But I have seen and written quite a bit of that sort of code. It's horribly unsafe and horribly unproductive to write. It's far worse than any C++ code I have ever seen. It's the only choice left when you have boxed yourself into a corner with a language that is unsuitable for the task.
>First, we pay in completely different ways for writing this software in manually-managed languages. Security vulnerabilities. Bugs. Development time
This is not a bad argument, but I think there has always been a very wide range of safety features in non-GC languages. C was never the only language choice. We had the Pascal family of languages. We had ADA. We got "modern" C++, and now we have Rust.
If safety was ever good enough reason to use GC languages for systems/infrastructure, that time is now over.
Go uses buffers pretty idiomatically and they don't seem unsafe or unproductive. Maybe I'm not following your meaning?
> If safety was ever good enough reason to use GC languages for systems/infrastructure, that time is now over.
I don't know that I want GC for OS kernels and device drivers and so on, but typically people arguing against GC are assuming lots of garbage and long pause times; however, Go demonstrates that we can have low-latency GC and relatively easy control over how much garbage we generate and where that garbage is generated. It's also not hard to conceive of a language inspired by Go that is more aggressively optimized (for example, fully monomorphized generics, per this article or with a more sophisticated garbage collector).
I think the more compelling reason to avoid a GC for kernel level code is that it implies that the lowest level code depends on a fairly complex piece of software, and that feels wrong (but that's also a weak criticism and I could probably be convinced otherwise).
Standard ML as implemented by MLton uses full monomorphization and a host of advanced functional optimizations. That code can be blazingly fast. MLton does take a long time to compile, though.
I've been working on a systems language that is GC'd and also does full monomorphization--Virgil. I am fairly miserly with memory allocations in my style and the compiler manages to compile itself (50KLOC) at full optimization in 300ms and ~200MB of memory, not performing a single GC. My GC is really dumb and does a Cheney-style semispace copy, so it has horrible pauses. Even so, GC is invisible so the algorithm could be swapped out at any time.
For an OS kernel, I think a GC would have to be pretty sophisticated (concurrent, highly-parallel, on-the-fly), but I think this is a problem I would love to be working on, rather than debugging the 19000th use-after-free bug.
Go's GC is very sophisticated, with very low pause times. It trades memory for those low pause times and can suffer from fragmentation because it doesn't compact memory. Concurrent copying is still a hard problem.
Again, a problem I'd rather we had than the world melting down because we chose performance rather than security.
There are higher level features that cause higher resource consumption. GC is clearly one such feature. No one denies that. So how do we decide where it makes more sense to use these features and where does it make less sense?
What I'm saying is that we should let ourselves be guided by the ratio development_time / running_time. The smaller this ratio, the less sense it makes to use such "resource hogging" features and the more sense it makes to use every opportunity for optimisation.
This is not only true for infrastructure/systems software. This is just one case where that ratio is very small. Another case would be application software that is used by a very large number of people, such as web browsers.
We've majorly screwed up our priorities. Correctness should be so much higher up the priority list, probably #1, TBH. When it is a high priority, we should be willing to sacrifice performance to actually get it. The correct question is not if we should sacrifice performance, but how much. We didn't even get that right.
But look, I know. Security doesn't sell systems, never has--benchmarks do. The competitive benchmarking marketplace is partly responsible. And there, there's been so much FUD on the subject that I feel we've all been hoodwinked and conned into putting performance at the top to all of our detriment. That was just dumb on our (collective) part.
Let me put it another way. Go back to 1980. Suppose I offered you two choices. Choice A, you get a 1000x improvement in computer performance and memory capacity, but your system software is a major pain in the ass to write and full of security vulnerabilities, to the point where the world suffers hundreds of billions of dollars of lost GDP due to software vulnerabilities. Choice B, you get an 800x improvement in computer performance, a 500x improvement in memory capacity, and two thirds of that GDP loss just doesn't happen. Also, writing that software isn't nearly as much of a pain in the ass.
Which did we choose? Yeah. That's where the disagreement lies.
Let's say the GC is a binary... where is this binary? Probably somewhere in the filesystem? So now you need a filesystem written in a language that doesn't depend on the GC, just like the binary loader, scheduler, bootloader, which by the way, can boot from remote, so now you also need a network stack that doesn't use GC.
You might need to show something on the screen before the GC starts up, so you will need video drivers, bus drivers, etc. written in a language that doesn't use a GC.
And in which language is the GC written?
You get the idea.
Programs in un-managed languages can be slow too, and excessive use of malloc() is a frequent culprit. But the difference is that if I have a piece of code that is slow because it is calling malloc() too much, I can often (or at least some of the time) just remove the malloc() calls from that function. I don't have to boil the ocean and significantly reduce the rate at which my entire program allocates memory.
I think another factor that gets ignored is how much you care about tail latency. I think GC is usually fine for servers and other situations where you are targeting a good P99 or P99.9 latency number. And indeed, this is where JVM, Go, node.js, and other GCed runtimes dominate.
But, there are situations, like games, where a bad P99.9 frame time means dropping a frame every 15 seconds (at 60fps). If you've got one frame skip every 10 seconds because of garbage collection pauses and you want to get to one frame skip every minute, that is _not_ an easy problem to fix.
(Yes, I am aware that many commercial game engines have garbage collectors).
Manual memory management on the heap does not solve this problem at all, it is actually worse at it than most traditional GCs. There is no magical system that allocates an object without executing code.
I will admit that the programming culture is different for many GC'd languages' communities, sometimes encouraging a very trashy programming style, which contributes to the perception that GC itself is the problem, but based on my experience in designing and building GC'd systems, I don't blame GC itself.
For some languages (I’m looking at you, Java), there’s not much of a way to program that doesn't generate a bunch of garbage, because only primitives are treated as value types, and for Objects, heap allocations can only be avoided if escape analysis can prove the object doesn’t outlast its stack frame (which isn’t reliable in practice.) (Edit: or maybe it doesn’t happen at all. Apparently escape analysis isn’t used to put objects on the stack even if they are known to not escape: https://www.beyondjava.net/escape-analysis-java)
I honestly can’t imagine much of a way to program in Java that doesn’t result in tremendous GC pressure. You could technically allocate big static buffers and use a completely different paradigm where every function is static and takes data at a defined offsets to said buffers, but… nobody really does this and the result wouldn’t look anything like Java.
Sometimes it’s appropriate to blame the language.
> Sometimes it’s appropriate to blame the language.
Oh, I know, I was just being vague to be diplomatic. Java being generally trashy has been one of the major motivators for me to do Virgil. In Java, you can't even parse an integer without allocating memory.
But the job of avoiding garbage is much easier in Go :)
The light-bulb idea here is about reliability, security and productivity. Managing memory manually a la C/ C++ is a proven disaster in all three respects.
Maybe ten or fifteen years ago GC was costly, but these days it's highly efficient and not incomparable with programmatic allocation.
However that doesn't make all GC languages equal and measured by Java's bad decisions at birth.
In userland I consider anything above maybe 1 or 2 milliseconds to be a long pause time. The standards only get higher when it's something like a kernel.
A kernel with even a 0.2ms pause time can be unacceptable when working with ultra low-latency audio for example.
https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i...
30% of CPU spent on GC, individual GC pauses already in the milliseconds, despite a tiny half-gig heap in 2019. For gamedev, a single millisecond in the wrong place can be enough to miss vsync and have unacceptable framerate stutter. In the right place, it's "merely" 10% of your entire frame budget, for VR-friendly, nausea-avoiding framerates near 100fps. Or perhaps 90% if "single digit milliseconds" might include 9ms.
Meanwhile, the last professional project I worked on had 100ms pauses every 30 seconds because we were experimenting with duktape, which is still seeing active commits. Closer to a 32GB heap for that project, but most of that was textures. Explicit allocation would at least show where the problematic garbage/churn was in any profiler, but garbage collection meant a single opaque codepath for all garbage deallocation... without even the benefit of explicit static types to narrow down the problem.
> So by simply reducing GC frequency, we saw close to a ~99% drop in mark assist work, which translated to a~45% improvement in 99th percentile API latency at peak traffic.
Did you look at the actual article? (Because it doesn't support your point). They added a 10GB memory ballast to keep the GC pacer from collecting too much. That is just a bad heuristic in the GC, and should have a tuning knob. I'd argue a tuning knob isn't so bad, compared to rewriting your entire application to manually malloc/free everything, which would likely result in oodles of bugs.
Also:
> And it's bullshit.
Please, we can keep the temperature on the conversation down a bit by just keeping to facts and leaving out a few of these words.
I did and it does for the point I intended to derive from said article:
>> However, the GC pause times before and after the change were not significantly different. Furthermore, our pause times were on the order of single digit milliseconds, not the 100s of milliseconds improvement we saw at peak load.
They were able to improve times via tuning. Individual GC pause times were still in the milliseconds. Totally acceptable for twitch's API servers (and in fact drowned out by the several hundred millisecond response times), but those numbers mean you'd want to avoid doing anything at all in a gamedev render thread that could potentially trigger a GC pause, because said GC pause will trigger a vsync miss.
> I'd argue a tuning knob isn't so bad, compared to rewriting your entire application to manually malloc/free everything, which would likely result in oodles of bugs.
Memory debuggers and RAII tools have ways to tackle this.
I've also spent my fair share of time tackling oodles of bugs from object pooling, meant to workaround performance pitfalls in GCed languages, made worse by the fact that said languages treated manual memory allocation as a second class citizen at best, providing inadequate tooling for tackling the problem vs languages that treat it as a first class option.
You might want to take a look at this:
> Sin two: explicit garbage-collection invocation. JavaScript does not have a Java-style System.gc() API, but some developers would like to have that. Their motivation is proactively to invoke garbage collection during a non-time-critical phase in order to avoid it later when timing is critical. [...]
So, no explicitly GCing when a game knows it's idle. Gah. The worst part is these are entirely fine points... and somewhat coherent in the context of webapps and webpages. But then when one attempts to embed v8 - as one does - and suddenly you the developer are the one that might be attempting to time GCs correctly. At least then you have access to the appropriate native APIs:
* https://v8docs.nodesource.com/node-7.10/d5/dda/classv8_1_1_i... * https://v8docs.nodesource.com/node-7.10/d5/dda/classv8_1_1_i...
A project I worked on had a few points where it had to explicitly call GC multiple times back to back. Intertwined references from C++ -> Squirrel[1] -> C++ -> Squirrel meant the first GC would finalize some C++ objects, which would unroot some Squirrel objects, which would allow some more C++ objects fo be finalized - but only one layer at a time per GC pass.
Without the multiple explicit GC calls between unrooting one level and loading the next, the game had a tendency to "randomly"[2] ~double it's typical memory budget (thanks to uncollected dead objects and the corresponding textures they were keeping alive), crashing OOM in the process - the kind of thing that would fail console certification processes and ruin marketing plans.
[1]: http://squirrel-lang.org/
[2]: quite sensitive to the timing of "natural" allocation-triggered GCs, and what objects might've created what reference cyles etc.
I mean, that is literally what the idle time scheduler in Chrome does. It has a system-wide view of idleness, which includes all phases of rendering and whatever else concurrent work is going on.
> Intertwined references from C++ -> Squirrel[1] -> C++ -> Squirrel meant the first GC would finalize some C++ objects, which would unroot some Squirrel objects, which would allow some more C++ objects fo be finalized - but only one layer at a time per GC pass.
This is a nasty problem and it happens a lot interfacing two heaps, one GC'd and one not. The solution isn't less GC, it's more. That's why Chrome has GC of C++ (Oilpan) and is working towards a unified heap (this may already be done). You put the blame on the wrong component here.
Roughly, but it has poor insight into a game's definition of "idle". Probably fine for most web games, but prioritizing, say, "non-idle" game-driven prefetching and background decompression over GC can be the wrong tradeoff.
> This is a nasty problem and it happens a lot interfacing two heaps, one GC'd and one not. The solution isn't less GC, it's more. That's why Chrome has GC of C++ (Oilpan) and is working towards a unified heap (this may already be done). You put the blame on the wrong component here.
Two non-GCed heaps doesn't have this problem, nor do two "GC"ed heaps if we use an expansive definition of GC that includes refcounting-only systems - it only arises when using multiple heaps, when at least one of them is the deferred scanny finalizer-laden GC style. While you're correct that "more GC" is a solution, it's not the only solution, it has it's drawbacks, and holding GC blameless here is disingenuous when it's the only common element. That these GCs compose poorly with other heaps is a drawback of said GCs.
If I try mixing Squirrel and C#, I'll run the risk of running into the same issues, despite both being GC based. I'm sure you'll agree that attempting to coax them into using the same heap is a nontrivial endeavor. I've been in the position of having two different JavaScript engines (one for "gameplay", one for UI) in the same project - but they were fortunately used for separate enough things to not create these kind of onion layers of intertwined garbage. While silly from a clean room technical standpoint, it's the kind of thing that can arise when different organizational silos end up interfacing - a preexisting reality I've been inflicted by more than once.
Sure. Let's avoid some of these words too:
> unsupported, tribalist, firmly-held belief that is unsupported by hard data.
Asking for examples is fine and great, but painting broad strokes of the dissenting camp before they have a chance to respond does nothing to help keep things cool.
> cherry picks worst-case examples and represents them as normative
Neither of my examples are anywhere near worst-case. All texture data bypassed the GC entirely, for example, contributing to neither live object count nor GC pressure. I'm taking numbers from a modern GC with value types that you yourself should be fine and pointed out, hey, it's actually pretty not OK for anything that might touch the render loop in modern game development, even if it's not being used as the primary language GC.
At no point in invoking https://en.wikipedia.org/wiki/No_true_Scotsman does one bother to define what a true scotsman is, only what it is not by way of handwaving away any example of problems with a category by implying the category excludes them. It's exactly what you've done when you state "People who argue against GC are almost always arguing against" some ancient, nonmodern, unoptimized GC.
Modern GCs have perf issues in some categories too.
> As such, it's inadequate to refute GC in general on the basis of long pauses and lots of garbage, you must refute each GC (or at least each type/class of GC) individually.
I do not intend to refute the value of GCs in general. I will happily use GCs in some cases.
I intend to refute your overbroad generalization of the anti-GC camp, for which specific examples are sufficient.
> Also, you can see how cherry-picking pathological, worst-case examples doesn't inform us about the normative case, right?
My examples are neither pathological nor worst case. They need not be normative - but for what it's worth, they do exemplify the normative case of my own experiences in game development across multiple projects with different teams at different studios, when language level GCs were used for general purpouses, despite being bypassed for bulk data.
It's also exactly what titzer was complaining was missing upthread:
> I've seen this sentiment a lot, and I never see specifics. "GC is bad for systems language" is an unsupported, tribalist, firmly-held belief that is unsupported by hard data.
Yes, that is how a NTS works, which is why you should be able to see that my argument isn't one. If I had said the common criticism of GCs ("they have long pause times") is invalid because the only True GC is a low-latency GC, then I would have made an NTS argument. But instead I argued that the common criticism of GCs doesn't refute GCs categorically as GC critics believe, it only refutes high-latency GCs. I'm not arguing that True GCs are modern, optimized, etc, only that some GCs are modern and optimized and the conventional anti-GC arguments/assumptions ("long pause times") don't always apply.
I've seen this sentiment a lot, and I never see specifics. "GC is bad for systems language" is an unsupported, tribalist, firmly-held belief that is unsupported by hard data.
On the other hand, huge, memory-intensive and garbage-collected systems have been deployed in vast numbers by thousands of different companies for decades, long before Go, within acceptable latency bounds. And shoddy, poorly performing systems have been written in C/C++ and failed spectacularly for all kinds of reasons.