My guess is, not many for AI to learn atm. [1] Students suffering from depression are 3x more likely to attempt and do so when they are less depressed. [3] US suicide rate is 14 per 100,000 (age adjusted and closer to youtube viewers). [0] Taking worst case 25:1 ratio for successful attempts. There would be 350 attempts for every 100,000. Let's say YouTube can somehow narrow down it to students suffering from depression (one category). 3x multiplier - 1050 attempts for 100,000. That's presumably only 1% of very specific targeted audience. You need to decrease that 1% to match people who will supposedly get triggered by a depressing video on YouTube. Conservative guess is 10x less so 0.1% and [2] 7% people of total population suffer from depression in US. Isn't that something around 0.007% of population?
What is the motivating factor for the algorithm? Profit. I just don't see how it is magically supposed to learn that. In fact, individuals suffering from depression are more likely to be profitable for youtube if it earns money based on consumption alone.
Disclaimer: this is a quick broad guess. It can be pretty wrong.
0] https://en.m.wikipedia.org/wiki/Suicide_attempt#epidemiology
1] https://www.ukessays.com/essays/psychology/depression-and-su...
2] https://www.samhsa.gov/data/sites/default/files/cbhsq-report...
Say that YouTube’s algorithm optimizes to offer viewers content that they would like to watch. This maximizes viewer platform engagement which maximizes advertisement revenue.
Say a clinically depressed person starts to watch videos on YouTube. The algorithms does what is does well. Optimize viewer retention and offers more depressing videos to keep the viewer engaged. Viewer gets more even more depressed?
What is the moral programming goals?
Same could be said for example of violent content. Viewer watches violent content get offered violent content?