The vast majority of commercial software development can use off the shelf native components and pair them with garbage collected languages these days.
Initially our Go app could only manage reading 10MB/s from Kafka, which is pretty sad. After switching from Shopify’s “Sarama” Go library to librdkafka (a Kafka client written in C), as well as some other arcane optimizations, we were able to 20x our throughput and handle up to 200MB/s, despite the overhead of interop between C and Go. On top of this, we used a library that wrapped librdkafka in Go bindings (confluent-kafka-go) and it turned out to have a much simpler interface than Sarama, in addition to it outperforming Sarama 20x.
While we didn’t end up rewriting the thing in Rust, we did end up leveraging a library written in C for major performance wins, in a case where scaling out was not possible. And let us also remember that even when scaling horizontally is possible, it is not necessarily even desirable in all circumstances: distributed systems, aside from introducing some amount of overhead, are basically Pandora’s box wrt. failure modes.
I've worked with Elasticsearch clusters indexing close to a million documents per second from kafka. That takes a lot of CPU, memory, bare metal, etc. and it taxes the JVM garbage collection quite a bit. But it can be done. We basically maxed out CPUs across 30 or so nodes. Basically, when you get close to maxing out the disk IO, you know you are doing a decent job.
People reach for C for a lot of reasons. And not always the right reasons. But if it works, it works. A big reason seems to simply be that interfacing with it from various other languages is pretty straightforward. That's why python developers don't worry about performance. They can always plug in something native if it becomes an issue.
Somehow it gets lost in the discussions as everyone associates their GC experience with Python or whatever.
GC mostly just works really well. Occasionally you get some poor performance and then you find that you are doing something silly and then you fix it and move on. Or you throw more hardware at the problem.
GC has a bad reputation but the reality is that there is a lot of software being written with languages that aren't exactly famous for even having a particularly good garbage collector. Python, ruby, go, etc. They are alright but certainly nowhere near best in class when it comes to garbage collection. If that matters to you, you can pick python and ruby implementations that have way better performance than the default implementations provide. But the thing is, it does not matter. It's genuinely not a problem for people using those languages.
I\ve used each of those languages and also use Kotlin both on the JVM and in the browser. So, you'd think I spend a lot of time obsessing about what the javascript garbage collector does in the browser or how to tune the jvm? Nope. I just let it do its thing. We have no garbage collect related issues. So why spend time on it?
My observation is that engineers love talking about optimization but rarely realize how expensive they are relative to just throwing more hardware at a problem. And the thing is, some modest hardware goes a long way these days. We run our Spring/Kotlin JVM servers on cheap vms in Google cloud and in another data center using openstack. 2GB and one VCPU. About 25$ per month. That's around 10 minutes of my time at my freelance rate. Those are European rates, not Silicon Valley rates. Those are more like 3-5 minutes. We could go cheaper and I'm pretty sure I can make it run with half an GB if I work at the problem. But the savings are negligible and I like a little safety margin. We have many issues but our hosting bills are pretty awesome. Cutting them in half is literally not worth more than 1-2 hours of my time per month.
I think it's great that C/C++ are getting some competition from modern languages now. Rust has a pretty innovative borrowing mechanism and it seems to genuinely be better at certain things that have been the exclusive domain for heavily optimized C/C++ systems for decades. That's great. Zig seems like the more reasonable choice for people that just want something simple that works that isn't ancient and awkward like C but gets you ballpark in the same league in terms of performance. I learned C a long time ago and I love Linux. But the never ending exploits and security problems are a thing that I just don't need in my own projects. Having a garbage collector and some other safety has a tiny cost but eliminates whole classes of bugs and exploits. Completely worth it IMHO.