I don't have research to cite about specifically whether novel recommendations get clicked more often. However, YouTube makes $15 billion in revenue. The recommender is responsible for an enormous fraction of that revenue. Google and YouTube have an enormous group of talented ML engineers. I am certain that they have tested changing the proportion of novel recommendations, and whatever proportion they use is the proportion that maximizes engagement.

Personally, I hate the YouTube recommender as well. It's particularly aggravating that I'm stuck with it and its lack of configurability even though I pay YouTube to make the ads go away, so they get no additional revenue when I watch more.

However, Netflix does many of the same things as YouTube, including autoplay, because they have found that autoplay makes you watch more, and if you watch more, you're more likely to renew your subscription.