Data analytics truly feels like a bubble.

Netflix has achieved the dream of movie studios going back more than a century now. They have the talent, the money, and more than two decades of data. Netflix knows what you watch, when you stop watching, how often you watch, which movie covers work best.

And yet, it's hard to look at Netflix as anything more than a total failure of the promises of data analytics and personalization. Netflix should be putting out nothing but hits. A dozen Breaking Bads or Game of Thrones.

Yet they are not. In fact, they do not even have a single show that is to the level of Mad Men, Breaking Bad, or The Wire. HBO and AMC are running laps around Netflix. Meanwhile, Netflix is making live action Cowboy Bebop and cancelling it before people even know it existed. I'm really curious what the data said about funding that particular project. On one hand, you have the cult following of the anime that will absolutely tear a live action version to shreds. On the other hand, you have to convince the uninitiated into viewing a remake of 23 year old anime.

Then there is the personalization. The fact that there is a meme about spending more time browsing the Netflix catalog than watching content tells you everything you need to know about how little people trust Netflix recommendations. Their new "top 10" feature is just depressing most of the time. It looks like a list of ten random DVDs in the bargain bin near the Walmart checkout line. Oh, and, their top 10 feature is currently the biggest recommendation feature on their site. And it's not even personalized! If that's not a complete admission of defeat I don't know what is.

I think Netflix has two big issues. The first is the way they drop seasons all at once prevents the natural cycle of pre-episode hype, post-episode interviews, speculation and leaks, fan anticipation, fan arguments (ship wars, etc), etc. Fan culture can't develop around this content as easily because there's never any breathing space for fans to collectively sit with the story so far. The shows that become a cultural force like Breaking Bad or Game of Thrones need us to keep coming back to the conversation every week. They have us talking about what happened last week and will happen this week with our colleagues at work and around the dinner table. How can these Netflix IPs enter stable orbit in the cultural zeitgeist when they're once-a-year events? It devalues the work, positioning it more like a movie that you watch and then forget rather than a story you become invested in over a long period of time.

Full season drops also allow people to binge a whole season of content for a single month's subscription and then immediately churn (ask me how I know). I assume they have some data-driven reasons for doing this, but it makes absolutely no sense to me.

Besides that, I think they should put more wood behind fewer arrows. They've developed a reputation for aggressively cancelling smaller shows with passionate followings which makes a lot of people not even want to bother until something has become an established mainstream success. I think of them now as the Google of content-creation, putting out a lot of solid (but not amazing) products and then cancelling them once people start to grow attached.

That's not how the entertainment business works. If I had to guess, Netflix's data-driven approach to content production is like card-counting in blackjack. It only gives them a slight edge on the house. A net positive outcome over hundreds or thousands of hands but offering no guarantee over the outcome of any single hand.

> HBO and AMC are running laps around Netflix

Netflix did win the most Emmys in 2021.[1] And they only started producing original content in 2013 or so. That's pretty good.

1. https://variety.com/2021/tv/news/netflix-emmys-the-crown-que...

They may also be optimizing for revenues as opposed to recommendation quality (homegrown content being cheaper than licensed)

But most people are lazy and won’t account switch for different contexts like that anyway, so there’s just no way they can keep the profile data as clean as it needs to be for a television.

This is exactly what their data analytics has told them to do.

Have a few popular, quality tv shows with star-studded casts as loss-leaders to bring in new viewers. Otherwise the model is to produce and recommend the shows that get them the most eyeballs per dollar; the bare-minimum to keep their subscribers there:

- Stand up specials are dirt-cheap, quite popular, and provide never-ending variety.

- Ditto with 'reality' shows, bake-offs, make-offs, expose documentaries, etc.

- Old sitcoms and b-movies that have a proven re-watch-rate.

Throw in a handful of first seasons to keep the FOMO up, and you've got a captive audience on the cheap. Maybe one or two will catch on and become the next loss-leaders.

They may not have the quality shows that are 'running laps around' HBO and AMC, but by any of the metrics Netflix cares about they are simply running laps around HBO, AMC and everyone else.

I don't follow this argument. Knowing what people like has very little to do with the quality of original creative content; surely you don't expect Goodreads to put out Shakespearean novels, or Spotify to be producing original hits on par with the Rolling Stones? Should ESPN have better pro sports scouting and coaching talent than the professional leagues?

Knowing what people like, however, _does_ have to do quite a bit with selling those people a product - which Netflix just reported 15% YoY growth to $7.7B yearly revenue, they're clearly very successful to this end. I think you actually have it backwards - if anything, Netflix represents a total fulfillment of the promises of data analytics and personalization. Despite mediocre original content, this is a $200B company with 200M subscribers growing revenues by double digits two decades after IPO.

If Netflix paid $450k+ salaries to screenwriters instead of engineers, you'd very likely get better movies on a worse streaming platform. And when Netflix has shelled out for Hollywood talent, like Mindhunter which has David Fincher and Charlize Theron, the results are quite good.

Regardless, to take the fact that Netflix pays for premium engineering and analytics talent, but does not pay for premium filmmaking talent, and then spin that fact into Netflix being "a total failure of the promises of data analytics and personalization" is a questionable criticism.

Ironically, it seems like AmazonPrime is far better at that.

As for then top ten being bottom barrel stuff, I think you overestimate how popular mad men was vs. something like king of queens.

I will say, Netflix seems to fail in many cases, and I don't understand how they think content discovery is supposed to work.

There may be others, but off the top of my head: Bojack Horseman.

The very high quality pieces of information can be things like "wants to buy a life insurance policy this week" or "just had a baby" or "just bought a plane ticket to XYZ," or "is in the frequent flyer program and spends more than $20,000 per year on travel."

However the majority of information about people, the overwhelming majority of whom have no significant disposable income, is worthless and not worth tracking for the most part. You reach those people through traditional mass marketing means.

And from what it seems to me, there is a giant bubble. The vast majority of companies doing "data science" jobs are things that a smart undergrad can do with a month or two of training. And this is because I believe C-suites have completely gulped down the data is oil mantra. There are entirely charlatan companies with unicorn, even decacorn valuations now being built on this mantra - for example, CRED in India.

Yet, as you said, and as I believe too, most of the data is worthless.

For example, I'm about to list my house on the market. The real estate agents who reached out to me last year and the year before that to try to induce me to sell the house had no chance of succeeding. Now is the time if they knew the secret that I'm about the list the house that they should all be competing for my business. The only relevant piece of information is that I'm about to sell the house. That is a valuable lead that many in my region would bid on. My background information is frankly not that relevant to the value of that lead, and isn't something that is readily surfaced by the kind of deep profiling that is supposed to be going on. It can be signaled by me signing up to some kind of list that sells my lead to a zillion people at once, but is never going to be surfaced accurately to the people who can earn the most profits from it by my Youtube habits or whatever.

Zillow sells the logged in user data in the market I'm buying in to the real estate agents listing the houses, but that again is not something being modeled by some kind of big data operation, but is merely the same kind of "little data" provided on things like dating websites or LinkedIn when people browse your profile. There's no modeling going on there that requires sophistication.

I've had a supposition for a while now that the targeted advertising industry should be closing the consumer cashflow loop by advertising effective self- and employment improvement, with the objective of increasing the disposable income people have so they can then make more brokering traditional sales

Personalized adverts and recommendations can be incredibly, horrendously dumb.

Here's what I see when I hit amazon's homepage at the moment : A "buy once again" column that features blackout curtains I bought 3 months ago (no, curtains don't need to be replaced every months, amazon.), USB cables I bought multiples of in the same time frame, a wireless charger (I already bought two before). An entire line dedicated to showing me backpacks (I bought one less than a year ago) An entire line dedicated to headphones (I recently bought wireless IEMs) An entire line dedicated to watches (same)

I don't get it. Supposedly the best and brightest work at firms like amazon and google to brainwash us to buy stuff, but classic, random, non-targeted advertisement is more likely to make me discover products I'd buy than targeted advertisement because the latter only shows me things after I don't need to buy them anymore!

Here's what I would expect actually intelligent targeted advertising to do : After buying a smartphone, recommend accessories (cases, screen protectors, USB-C dongles, chargers, whatever) Here's what targeted advertisement actually does : show me smartphones ads everywhere I go after I already selected and BOUGHT a smartphone. No, I don't need to buy another smartphone weeks after a recent replacement, amazon!

The same sort of phenomenon can happen after google locks on searches I did to buy something. I can't wait to see the internet advertisement industry crash and burn, it's overvalued nonsense.

Disagree there. About 75% of the things I buy on Amazon are repeating purchases that I nevertheless don't want to be automatically scheduled. It used to be a real pain in the neck to reorder something manually, so I'm glad they made that easier.

But yes, in general, Amazon is full of low-hanging fruit that's been neglected on the tree for a decade or more. Buying clothes from Amazon still manages to be a worse experience than going to the mall, for instance, which is really saying something.

Both YouTube and Amazon heavily personalize by recommending primarily the 3-4 things that I've interacted with in the very recent past.

Make it fancy and use a multi armed bandit and call it machine learning/AI/data science.

So the question is this: if they're not (and never were) using that data for what they say they were, what are they doing with it?

The most common pattern I see relating to personalized advertising as someone being advertised to is that I will often see an ad for something I just bought (or some competitor to it) repeated relentlessly for a couple of days after buying it and this is after not seeing any related ads during the days prior where I was actually doing some research into the product space.

Maybe I'm an outlier but they seem to miss the window of relevance on me often enough that I notice it as a commonly repeated pattern.

A return rate of say 1% may lead to more people looking to buy a drill who have just bought one in the last week than people looking to buy their first drill.

If that's true, they left something out of their calculation: What percentage of people will install an adblocker as a result of feeling like they're being hounded for a few weeks? This scenario was mentioned specifically by Tim Cook when he introduced the Safari anti-tracking features.

So that you can continue to show me ads for a washing machine for months after I purchased a washing machine.

Apparently now I'm into baseball, flowers, boating, celebrities, country music, credit cards, geology, event ticket sales, fishing, and windows OS. Among a couple hundred other things. It even gets some rather basic facts (marital status, company size, education) wrong. I seriously wonder how they generate this profile?

https://adssettings.google.com/

Poorly!

At the time I was a late 20s, college educated, South Asian male. I’m very cis and very straight. And yeah my musical tastes are pretty eclectic, but that was a weird profile to settle me on.

Better to just follow people's interests, instead of using their interests to incorrectly pigeonhole them and then drawing incorrect generalisations from that.

Something about my actual interests and activity apparently makes youtube think I'm into Fox news and all the crazy shit found there.

Now, who else has this same value judgement about me? This assessment that I neither declared for myself nor even ratified.

It's annoying but ultimately harmless that youtube shows me conservative wackjob stuff.

But is that same profile in someone else's database that marks me as someone to watch or something? Does it affect my insurance rates, my liklihood to get extra scrutiny when travelling, my ability to purchase or register a firearm, my access to jobs that might be extra sensitive or responsible, basically any of the things where someone either private or the state does any sort of background or credit check on you for any reason, and there are really many of those when you think a out it.

I'm guessing, today, it's probably not really affecting my life in any real way, but, there is no way it makes any sense to say that will still be true tomorrow.

I spent a lot of time learning about association rule mining in my AI courses, including the implementation details of competing ways to mine them. The technique seems extremely useful and fascinating (I jury rigged it for on the fly league of legends champ recommendations to maximize calculated win rate change given limited information), but I almost never see it used in the real world or even see it talked about anymore.

What happened to association rule mining?

Personalized page rank is not significantly better at summarization in my experience, even "queryable" summarization, but that also could be a pure implementation problem or a problem of hyperparamater selection...

Then, after that, what's the point in showing drill ads to me for two weeks?

Probably that's what they are going for if they're doing it on purpose.

Do you have a link for further reading on that? That's fascinating if true.

It's pretty straightforward to understand that when the vast majority of your sites income is generated from ad revenue, that data is being used to optimize for generating ad clicks, etc., rather than actually giving users the best/most relevant/useful/desireable information for their purposes.

It seems to be completely useless for anything else, and specifically harmful for product discovery, that is the one way ads add societal value.