So, rather than being a deliberate choice of Google, it's simply an emergent phenomenon of a system Google expressly designed to manipulate people?

How is it not a hundred times more terrifying than the conspiracy theory, that google chooses to keep such a system online?

Oh, it's far more terrifying that the natural result of being good at feeding people things that you can determine that they will respond well to is that you will progressively feed them more extreme content, because tragedy of the commons is much scarier than mustache twirling villains.

At the point when you recognize this and then choose to proceed apace, you become a “mustache twirling villain.” YouTube crossed that threshold a long, long time ago.

Because our systems are already optimized in a hundred different terrifying ways.

There is a review posted around here of the book 'seeing like a state' that talks about how the modern world is not optimized for human needs, but the needs of the state to collect tax. It makes many good arguments on why it is the case.

Google didn't "expressly design" YouTube to manipulate people, come on, that's a wildly uncharitable view. Virtually all popular content hosting sites have some form of recommendation system - because people like them. Claiming it's all a conspiracy to manipulate people is the same school of thought that says all advertising is manipulation, all businesses are run by psychopaths, etc. It's fairytale stuff.

Please do watch the TED talk by Zeynep. She actually says something along these lines - that it's not so much a conspiracy but probably an unfortunate side effect. What the system ultimately cares about is that people stay longer on a site if they're fed more extreme content.

From my brief experience with neural networks, they work very well for some kinds of tasks but you never know why they calibrate themselves this way. One of uses of neural networks that impresses me the most is Keldon AI, the free (open source) app for Race for the Galaxy board game. Traditional game AI is notably bad with "fuzzy logic" - when you're unable exhaustively calculate everything, and your opponent is just getting some cards. Well it turns out machine learning doesn't care about that. It cares that if it has X cards in hand and you have Y cards in hand, and the table looks so and so, playing card Z has C chance of landing a win in the long run. Or something like that. And it works! But if you look inside the rftg app, you'll just see piles of numbers. Just weights. SO when you play against it, you see what it played, but you're never sure why, and it can't explain itself. Zeynep actually talks about this, but in more sinister context. For instance deep learning algorithms are eerily good at detecting bipolar people. They can take advantage of people with various mental disabilities and disorders. Think of the consequences. What if this is used to create your personal profile and share it across shops you visit.

I don't really see why this is a conspiratorial hypotheses. The goal of youtube is to keep you on youtube watching videos with ads. That is what they are trying to do, and it happens to coincide with pushing extreme content as an unintended side effect.

Exactly. No one here, least of all Tufekci, is claiming that YouTube has deliberately engineered their recommendation system to favor extreme content. The argument is that it's an unintended consequence of how the YouTube recommender's objective function interacts with human psychology: (1) YouTube trains its recommenders to maximize time on site. (2) It's a sad fact about humans that sensationalistic or extreme content is highly engaging. (3) YouTube's recommendation system thus learns to serve up sensationalistic and extreme content.

To have an algorithm that favors extreme content above non-extreme content because it keeps people hooked is indeed unlikely to be programmed directly.

I actually don't think it's learning personal preferences like you suggest either.

It likely is a collaborative filtering method. You get served what people "similar to you" have watched most. Similarity is based on your watching behavior. If you watched 1,2,5,6,7 and someone else 1,2,6,7,8. You will be recommended 8 as well.

It would be interesting to come up with an algorithm that suggest favorites of subgroups of viewers you don't belong to.

As an interpretation of people who worked on the algorithms:

> The YouTube algorithm I worked on heavily promoted Brexit, because divisiveness is efficient for watch time, and watch time leads to ads.

https://www.twitter.com/gchaslot/status/1036323806242066432

Not clear what exactly technically he is referring to, whether your explanation fits. I suppose an algorithm that tended to cluster people together could be described as divisive.

You know, when I read the tweet and the discussion I think, people worry about AI becoming paperclip maximizer, but this is worse.

Our own society is becoming paperclip maximizer! We want to spend more time watching interesting videos and more dopamine shots.. And the company and people working for it, even indirectly, oblige.

We worry that there will be an AI where all of what it does is a consequence of giant misunderstanding of human nature. But what if there is nothing to misunderstand about humans, but it's rather our own design flaws that cause our desires, when being taken as an objective function to be ruthlessly maximized, lead to society that is, at best, extremely dull.

> It likely is a collaborative filtering method. You get served what people "similar to you" have watched most. Similarity is based on your watching behavior. If you watched 1,2,5,6,7 and someone else 1,2,6,7,8. You will be recommended 8 as well.

Yeah, similarity of “preference” is approximated, in any recommender system, by this or some similar analysis of behavior with respect to content.

> unlikely to be programmed directly

In fact, YouTube is very left-leaning internally - if they were deliberately biasing content, you’d expect it to be biased in the other direction.

That's essentially what she believes as well. The only intent she ascribes to Google is a desire to keep people on the site, which, of course, is that point of any recommendation engine.

How is tying to "make people spend as much time on a site as possible" is a conspiratorial hypothesis?