I may have rewatched maybe 1% of the videos I've seen, so either I'm way outside their target group, the recommender isn't adapting to my behaviour, or 1% is the best it can do.

Also, based on my experience with GDrive support I really don't believe that Google has excellent engineers all over the place. Like virtually all big companies before them, once they get big enough the overhead of keeping the services from regressing takes up all the available energy.

That doesn't really explain why they re-recommend previously acted-on videos though. You won't watch them anyway so they basically eat screenspace that can be used for some novel videos with ads that you will watch with a higher probability, there is no real lack of those now.

People re-watch videos. If it's showing up in your recommendations, the recommender thinks you may act on it.

And youtube knows exactly what videos they re-watch often enough so it can clearly mind this stuff, plus they will probably use the search if they want to re-watch something specific anyway. It is no reason to re-recommend every one of those including explicitly uninteresting.

Or am I missing some research that shown novelty less effective at driving the clicks here?

I don't have research to cite about specifically whether novel recommendations get clicked more often. However, YouTube makes $15 billion in revenue. The recommender is responsible for an enormous fraction of that revenue. Google and YouTube have an enormous group of talented ML engineers. I am certain that they have tested changing the proportion of novel recommendations, and whatever proportion they use is the proportion that maximizes engagement.

Personally, I hate the YouTube recommender as well. It's particularly aggravating that I'm stuck with it and its lack of configurability even though I pay YouTube to make the ads go away, so they get no additional revenue when I watch more.

However, Netflix does many of the same things as YouTube, including autoplay, because they have found that autoplay makes you watch more, and if you watch more, you're more likely to renew your subscription.

Making big revenue and having best teams doesn't mean they do everything most efficiently and don't ever go the way of the lowest friction. I am really curious whether someone really tested it and found it unfeasible from a technical or psychological standpoint. The autoplay stuff I clearly understand but the recommendations' - don't.