On search, I wonder if this is deliberate; if they gave you exactly what you want as the top result, you'd click that one and not look any further. But by having to scroll past a few other videos, maybe you'll see another one or two that you also decide to click on, resulting in more overall views.
This is a different strategy to Google web search, where the monetisation occurs on the search results page (via ads there) rather than on click-throughs, so there isn't an incentive to try and get you to click several tangentially related links.
I also still don't understand why yt recommendations are as poor as they are. I've tried taking the time to try removing certain things from the "recommendation history" (or whatever they call it), flag recommendations I'm not interested in etc and it does improve things a bit but still it recommends things I would never watch in a million years (eg right now I have 4 Joe Rogan recommendations on my homepage even though I've literally never watched any of his things and have zero interest in ever doing so, and click on every one and say not interested etc, for a while it was obsessed with trying to get me to watch MMA content even though I can't bear watching other people hurt each other).
And the whole issue of recommending videos where it knows I've watched the whole thing, I just don't even know where to start.