The algorithms behind recommendation engines are well understood. Companies can and do invest huge amounts into making their algorithms better and it yes, it definitely takes a lot of effort to tweak and refine the algorithm to make the overall experience addictive and relevant.
But the seemingly widely held idea that TikTok builds better recommendation algorithms than other companies (including Facebook's News Feed) doesn't seem to stand up to scrutiny.
Also just letting YouTube keep automatically moving on to another video, it pretty quickly gets off into weird territory. It's like YouTube thinks that for any video you watch, you want to hyperfixate on that particular topic and are perfectly happy to end up in some weird video with hand-drawn badly-voiced Sonic and Mario characters having a tea party.
Or perhaps my favorite genre of "where did this come from?" is a series of stop-motion videos with figurines of Ash, Misty, Brock, and for some reason Link (from Zelda), sitting in a classroom, with Nurse Joy as the teacher, who puts up a silhouette of a pokémon on the blackboard and asks what it is, and the characters try to guess. It always ends up with something weird like a silhouette that looks exactly like Voltorb, and once everyone gives up, Nurse Joy reveals it as a top-down view of Jigglypuff. This one is weirdly entertaining. But also just plain weird.
Not quite as good but even weirder is the 3D animated but badly-rendered and badly-animated scenes of lego pokémon with wizard hats and wands that conjure up lego buildings and collaboratively build a farm.
---
I suspect the real difference here is TikTok wants to drive user engagement. It wants me to actively enjoy the videos I'm watching, put likes on them, maybe comment on them, share them with people, make my own videos inspired by them, etc. It wants me to be an active participant in consuming and creating content, and it wants me to always be confident of finding new content I enjoy by going through the feed. And me swiping past a video that I don't care for is perfectly fine as long as I don't have to do that very often.
YouTube on the other hand wants a passive viewer. It wants me to just sit there on the couch slowly going numb as I let it show me videos. If I never touch the remote control, that's a win for YouTube. Every time I hit the menu button to go back to the content list, that means YouTube failed. Picking new unique content is therefore a big risk for YouTube, its best bet is to just stay closely related to whatever it is I just watched, and to slowly migrate towards anything that keeps me fascinated. This means it tends to get weird.
Did you completely change your IP address?
Did you spoof/hide any other aspects of your browser configuration?
Private browsing or not, you would probably easily be recognisable based on IP + browser fingerprint alone so youtube could just continue to serve you the same videos.
It has completely updated based on what YouTube has learned about your interests based on your search and view history.
It works even without searching (eg, click through on a link on the default homepage).
I'm not convinced this is true. TikTok's algorithmic "For You Page" is astonishingly good. Somehow TikTok manages to zero in on stuff relevant to users extremely quickly, and without relying on spamming a lot of general interest popular content. I created a TikTok account recently and literally within minutes I was getting an extremely personalized FYP (For You Page) without even following anyone. If my wife and I compare our FYPs we'll find virtually no overlap (possibly a few cat videos but that's it). I've never seen anyone else do an algorithmic feed even half as good as this. Everyone else seems to rely on just spamming you with general-interest popular videos and hope you'll find enough people to follow to fill out the feed yourself. TikTok's FYP isn't even driven by follows, that's a separate feed.