I’d say it’s more likely they have super advanced/clever ways of doing the latter. The algorithm could be a simple dot product and the result could be great or terrible depending on how good the feature extraction is.

Pulling useful features out of videos is no small task. The fact that everyone raves about how good the recommendations are indicates to me that this is where their innovation lies.

There's so much good meta-data (likes, comments, duration, sound used, views, like/view ratio, skips, loops, subscribes, etc.) that I'd be surprised if they were digging into the contents of the video at all right now.

Bytedance has thousands of the smartest data scientists in China.

Bytedance has thousands of Manual Labor specialists as well.

Using ML it is very easy to tag videos.

They could also be digging only into audio, doing speech recognition on it, then clustering the text. Augment that with the text users have put into the video directly using the in-app editor and you have some pretty solid data.

If that were true, it'd be interesting to see if they push out support for close-captioning. It's an accessibility push, but also would leverage a lot of the same capabilities...

I would also start doing image recognition in the video frames, to extract things like gender, objects, etc.

Would this have any advantage over just using video embeddings (or a sequence of frame embeddings?) which in theory should capture those things in vectorized form.

> I'd be surprised if they were digging into the contents of the video at all right now.

Why would you be surprised to learn TikTok is doing video content analysis?

It can be a) very expensive b) also very difficult to implement.

Video understanding is an active field of research and I'm not sure state of the art is there yet for capturing nuance like engagement potential, categories etc.

State of the art where? College? Silicone Valley? Bangalore? Shanghai? Beijing? Hangzhou?

State of the art in academia, which is largely location agnostic.

Google was able to build a very useful search engine that ran for decades relying on the significance of links and keywords, without much understanding of the meaning of page content. You can get very far with the readily available data, before you need to delve into the fancy stuff to make it a few percent better.

They claim to be looking at the music in the video and avoiding sending you to another video with the same music.

That would be the "sound used". The music in the video is specified/labeled before upload so there's no need to actually process the sound of the video.

Almost all of those applies to YouTube, do they not ?

IIRC youtube vids are too long to do any useful feature extraction from the videos.

The comment I was responding to mentioned a lot of metadata around videos, that is what I was responding to.

More data beats smarter algorithms any day, and TikTok gets a lot of data because its videos are so short and interaction rate so high. There are tons of signals it can use as inputs: How much of a video did you let play before swiping next? How many times did you let the video loop? Did you like? Did you comment? Did you like a comment? Did you click through to the profile? Did you view other videos from the profile?

I don't buy this argument. YouTube ostensibly has a metric ton of information like this and even if tiktok has more training data now, I'm fairly sure their training data in principle was smaller than what YouTube historically had given their decades long presence and their ubiquity in the internet.

This is on top of the documented effort by YouTube to perfect their recommendation algorithm using the best ML minds they got [1] only to polarizing response from its users.

Clearly tiktok has other advantages (homogeneity in some content characteristics, viz. Extremely short videos which probably correlates to their simplicity) and has clearly tuned a fundamentally better recommendation algorithm that even the minds at Google brain couldn't figure out.

[1] https://www.theverge.com/2017/8/30/16222850/youtube-google-b...

YouTube has the problem that all Google products have: they put you in a filter bubble which you can never get out of. The algorithm also optimises for more "long form" content and it's pretty well known that the optimum video length is around 10 mins.

if you simply erase you watch history that will reset your recommendations, always works.

I just watched the sumptuous Gandhi movie and thought "Nice, now I'll search for some movies about the real Gandhi". After typing "Gand" in the remote control, there was Gandhi in the 5th, after searches about a more important historical figure: Gandalf.

The first video was titled in big letters "Was Gandhi a racist who spanked women?".

Make no evil. Duh!

YouTube's recommendations are arguably just as good as TikTok's in my opinion. The only difference is that YouTube places the burden of choice on you while TikTok makes every choice for you.

If you don't quickly find something you'd like to watch on YouTube you're very likely to leave and find something else to entertain you. Meanwhile TikTok is automatically feeding you an infinite source of quick and easy to digest content, all of which you'll probably like to some degree.

YouTube could do something similar and give users an automatic continuous feed (ala a TV channel), but I think YouTube's content is much too longform to work well in this format. This burden of choice problem affects Netflix in the same way which also has superb recommendations.

I also agree that YouTube's recommendations are pretty good. For example, I like to watch Linux distro reviews which are fairly niche and usually only get a few hundred views but my YouTube homepage is filled with them. In contrast, when I was testing out TikTok it mostly recommended me things with broad appeal like comedy skits and generic science videos that I think 90%+ of the population would find at least somewhat enjoyable.

> I also agree that YouTube's recommendations are pretty good. For example, I like to watch Linux distro reviews which are fairly niche and usually only get a few hundred views but my YouTube homepage is filled with them

This is exactly its main weakness. Once it finds one or two things you like, you're hardly ever going to see anything diversity, and if you ever do, it won't be long before that thing completely fills your feed.

I have two Google accounts, and I switch between them for non-YouTube-related reasons. The one is currently all Periodic Videos all the time, the other is all History Matters all the time. It's not as if I lose all interest in chemistry if I switch accounts. If I have a few minutes to kill, I'll end up thinking "History Matters isn't exactly what I want to watch, but I don't want to spend ten minutes trying to find something else", so I watch it, and the algorithm pats itself on the back and cues up another one of the same.

I never thought of it like this. Just had one of those mind exploding moments as I realized the frequency of interactions on TikTok vs other platforms and the virtuous cycle it creates (order of magnitude more data -> better recommendations -> even more interactions). Thanks for this.

Glad I was able to help introduce you to that idea. I was first introduced to it by Peter Norvig (Google): https://static.googleusercontent.com/media/research.google.c...

> Did you click through to the profile? Did you view other videos from the profile?

Probably why multi-part videos are so popular (you have to click their profile then find the part 2 video to finish the story they were telling).

Sounds like a way to game the system.

This article was popular here when it was published. https://www.eugenewei.com/blog/2020/8/3/tiktok-and-the-sorti...

They always get a response to every video that you start watching, making their training data much, much better than that of Facebook or Instagram.