Netflix claims Fortnite is now a bigger competitor than HBO

Re:Netflix claims Fortnite is now a bigger competitor than HBO by blobster on Sunday January 20, 2019 @01:42AM (#18951554)

I always wondered how some companies can rely exclusively on data like A/B testing to make decisions. Isn't that incredibly slow? In my experience there are thousands of decisions to make every year in a high growth company and there is just not enough time to test everything. This is why the best software companies that we are all familiar with revolve around an individual with incredible intuition/vision. If it was possible/practical to just rely on a bunch of A/B tests, every company would be successful.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by kbyatnal on Sunday January 20, 2019 @01:58AM (#18951589)

Not really. With the amount of users these companies have, you can get statistically valid results very fast (and you can run hundreds of tests in parallel).

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by masonic on Sunday January 20, 2019 @02:17AM (#18951636)

Then why is their search such utter crap?

When I search on a specific title, I get anything that roughly matches any keywords in combination, then also any title that remotely associates (even conceptually):in any possible way with any individual keyword.

But what it won't tell me is "we don't have the title you want".

Specific example from earlier: look for the 2007 film "Sunshine". Then, go plug "sunshine film" into duckduckgo and see what comes up as the very first item.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by jdietrich on Sunday January 20, 2019 @06:18AM (#18952501)

>But what it won't tell me is "we don't have the title you want".

That's by design, not accident. The Netflix catalog is much, much smaller than most people think, especially for films. They have just under 6,000 titles available in the US, but a large proportion of that is cheap filler that most viewers will have no interest in. Searching the catalog using a third-party tool reveals just how thin it really is.

https://unogs.com/

Netflix have focussed on recommendations over search because it makes their service look better. If you search for something, the odds are that Netflix don't have it. Recommendation allows them to promote their original content, which is Netflix's main moat - if you're hooked on Stranger Things or Orange Is The New Black, you're far less likely to cancel your subscription.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by dewey on Sunday January 20, 2019 @03:55AM (#18952025)

Do you really think that’s not on purpose or surprising? If you show them recommendations instead of an empty search result they might want to watch something else or get inspired to use other search terms to find some other result.

That may sound like a bad search for engineers but for the user it’s probably what they want and what Netflix measured.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by hombre_fatal on Sunday January 20, 2019 @07:32AM (#18952746)

Being an engineer is no excuse for not understanding basic product design nor being unwilling to see/accept how a feature benefits users.

(I'm saying that to agree with you, not to correct you)

This example is particularly good because they didn't even say "I get it, but I don't like it" or "I think it'd be better if..." Rather, they went straight to "wait, netflix devs are supposed to have skillz?" and "this must be incompetence", an assumption that represents one of the more toxic parts of our community.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by shalmanese on Sunday January 20, 2019 @03:12AM (#18951865)

I think the way they do it is very elegant. They surmise the film you want and if they don't have it, they put it in the top bar as "find films related to X". That way, they can push you into consuming something else even if they don't have what you were originally looking for.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by masonic on Sunday January 20, 2019 @04:24PM (#18955689)

   "find films related to X"

But my point is that most of the results are in no way related to X.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by groestl on Sunday January 20, 2019 @02:27AM (#18951673)

Probably causes some customers to watch into some of the crappy search result titles, to see if it's the thing they wanted. Hence, the "hours watched" metric goes up.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by homero on Sunday January 20, 2019 @02:21AM (#18951653)

That upsets me too. They hope I'll forget what i wanted and use a recommendation but I'm left confused and leave the app.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by mongol on Sunday January 20, 2019 @02:25AM (#18951663)

Let me guess: It has proved to work well for retention.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by masonic on Sunday January 20, 2019 @02:30AM (#18951682)

Probably not so much toward retention (I'd have quit over this were I not getting it free) as toward redirecting you to content they already control at little or no variable cost.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by blobster on Sunday January 20, 2019 @02:04AM (#18951604)

Maybe, but I'm talking about getting to that level in the first place.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by nostrademons on Sunday January 20, 2019 @02:23AM (#18951657)

Startups that are still searching for product/market fit don't really do much A/B testing - the user populations are too small for anything statistically relevant. They rely more on intuition, or as some would call it, blind luck.

In some ways, what an entrepreneur does is very coarse-grained A/B testing anyways. If you launch 50 different startup ideas and they all get zero traffic, and then you launch the 51st and it has 100 users sign up over a weekend, then there's a good indication that you should go with the 51st idea.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by quanticle on Sunday January 20, 2019 @03:38AM (#18951949)

It's also what venture capitalists do at the level of companies. Fund a hundred companies, and then watch them. Pour more money into the ones who appear to be succeeding.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by mehrdada on Sunday January 20, 2019 @02:03AM (#18951600)

I doubt there are companies exclusively rely on A/B tests to make decisions, but even if there were, someone has to think about hypothesis and design the right experiment to then confirm with data. You can't just have a computer running around and make experiments and confirm them for youº.

º: you kind of can, if you have enough fresh data, to search for and derive statistically significant results automatically.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by blobster on Sunday January 20, 2019 @02:08AM (#18951615)

Maybe it's just me, but I don't think the secret behind this kind of success is just properly designing experiments. Again, if this were true, the best founders would be data scientists. But they're not. They are visionaries. So I reckon it must be the vision.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by morningcovfefe on Sunday January 20, 2019 @02:13AM (#18951629)

Vision is obviously necessary - but a big problem folks run into is failing to recognize when something is not working. A/B testing is one way to keep yourself from falling into the trap of thinking 'I know better than my users what they should want'

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by TylerE on Sunday January 20, 2019 @02:48AM (#18951759)

Also a good way to find a local maximum when there are much higher maxima not far removed.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by blobster on Sunday January 20, 2019 @02:20AM (#18951647)

I agree, but I think this doesn't apply to innovation. When is comes to innovation, people always want faster horses.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by rv-de on Sunday January 20, 2019 @02:37AM (#18951718)

that depends on the validity and accuracy of your test metric.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by andrepd on Sunday January 20, 2019 @05:40AM (#18952375)

>Again, if this were true, the best founders would be data scientists. But they're not. They are visionaries.

[citation needed]

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by lenzm on Sunday January 20, 2019 @04:22AM (#18952129)

You'd be paralyzed by indecision if every decision had to be driven by the results of a test.

[ Parent | on HN ]

Re:Netflix claims Fortnite is now a bigger competitor than HBO by revicon on Sunday January 20, 2019 @06:26PM (#18956291)

Remember that Netflix has huge amounts of traffic. They get enough data from a/b tests deployed to only a tiny fraction of their users.

[ Parent | on HN ]