I have no doubt that total time spent in the app is a metric they track. Plus
scrolling through all of those videos on the way to the video you're actually looking for means there's a decent chance you'll see a thumbnail or title that catches your interest leading to more time watching videos. Either way, It'd be a shitty move and one that somebody should have caught and corrected a long long time ago.
Time spent scrolling is not time spent watching ads. So that is definitely not a metric YouTube is optimizing for.