By knowing the version, you can more efficiently determine which CVEs may apply, surely?

The arsenal of attacks includes "let's try and figure out server version", too.

Because if you know the version, it gives you a shortcut, and you can immediately switch to a smaller arsenal designed for that specific version.

On an individual basis it's sometimes hard to see the harm in this. But more specifically the problem with version identifiers is that attackers can scan across many servers, iterating which ones have a certain vulnerable version.

While a bit of an extreme scenario, I can imagine it being useful for high-level actors with their own private vulns which they'd like to keep on the down low unless absolutely necessary.

Exploiting a memory corruption vulnerability depends on the exact details of the software version and platform. Attempting the wrong exploit risks crashing the process, which might be noticed and investigated.

Not sure why you're being downvoted. Your question is reasonable and any attacker worth their salt will fire a full arsenal of all the CVE's at your environment regardless of version.

Hiding version numbers etc is an example of "Security through obscurity" and is a weak defence strategy:

https://en.wikipedia.org/wiki/Security_through_obscurity

An attacker worth their salt will most definitely not fire all their CVEs at risk of crashing the target process or leaving it in an undefined state.

And your second point is debatable. See defense in depth.