Uh, spam? Spam has a pretty damn massive bag of money attached to it and nobody has managed to get it right. Every single spam filter catches a ridiculous amount of false-positives and spam is far easier to classify than what's required here. Youtube catches a ridiculous amount of comments as false positives in their spam filter. I can't imagine this new category is going to be any better.
Not to mention quite a few false negatives. And the stakes on this are possibly higher, so it will be tuned toward an even higher false-positive rate.
The existence of the gap also doesn't change the fact that my experience is that implicit in the majority of the mainstream conversation about these topics is is an imagining of tech giants as wielders of unlimited power over their billions of users, and I'm skeptical that they'd be satisfied with most/any of the points in the valley you describe.
Not to mention that the flipside of public opinion is that the caprices of an algorithm are even more hated than their cases of leniency! All of the suggestions you make are going to increase false positives, and the PR backlash I've seen from that is far worse than the backlash to unmoderated content.
That isn't to say that there isn't room for reasonable complaint, but most people are really, really stupid, and even worse, entirely unconcerned with the consistency of their objections (a problem exacerbated by the fact that different vocal subgroups complain about different problems, and our various media will happily signal-boost all of them). Again, I don't see a solution that threads the needle effectively other than blunt restrictions, leaving everyone worse off but neutering the worst of the PR damage.
I think that the reality is that until very recently there wasn’t a huge bag of money attached the having fantastic moderation tools. I don’t moderate a large YouTube channel, but I will go out on a limb and guess that the tools are rudimentary.
Basic things like scoring the comment poster as well as the comment itself, the ability to have verified commenters (but why should this have to be a manual switch?), the ability to adjust the sensitivity of the classifier based on the intended audience of the video...
There is a massive valley of opportunity in between what we have today, and “magic wands to enforce arbitrarily nuanced moderation policies at scale” and perhaps the biggest comment platform on the Internet just decided they need to turn off comments entirely and if you do turn them back on, you need to do manual human pre-approval on each one.