My point is that you can start out without a GC and do a series of sane and effective optimizations that end you up with a GC. Just as you can start with a GC and optimize by moving more and more of your allocations to non-GC memory and end up without a GC. Which endpoint is faster depends on the workload.
You can say the same about any abstraction, such as having a generic memory allocator, using a standard library, actually about any code reuse. For example, many programs use printf, but don’t need all its features, so hand-rolling a custom version may lead to smaller, faster code (things get complicated here on systems with shared libraries)
The question always is whether lost performance (execution time, memory usage) is worth faster development.
GCs are useful in the same way that Python is useful even though it is slow. Simplicity of use has intrinsic value but that is a tradeoff against other objectives.