Here's a translation that may be useful: Just because you can't measure your epistemic error (error in your knowledge, worldview, assumptions, or anything else) doesn't mean it's not there.

In my experience it's the #1 source of errors that smart people make.

I assume this is getting downvoted because of its tone, but the sentiment is correct. The statement is nonsense. There can be wisdom in attempting to understand why anecdotes disagree with data, but the vast majority of the time it is simply a combination of selection bias and the human tendency to want to believe things that support your own worldview.

Bezos is making a subtle point here that I feel people are overlooking.

1. Anecdotes that you notice, by their nature, tend to come from reasonably sized groups of people.

2. "Data" as the term is normally meant in business and government does not. It may originally consist of many individual data points that represent something about large groups of people, but by the time it reaches a decision maker it has been filtered through normally just one or two people who have selected it, normalized it, and analyzed it in various ways.

In other words, "data" is often not being used with its dictionary definition of a raw CSV file but rather it means "analysis". And unfortunately the nature of data analysis is that not only is it easy to make genuine mistakes but it can also be very easily manipulated in all sorts of ways to present whatever the analyst wants to be true.

Academic research is an especially prevalent form of this problem because people don't hesitate to claim they've made a "data" based decision then cite an academic paper that presents a model with some absurd assumptions, or where the model outputs aren't visibly connected to the paper's conclusions at all, or which shows signs of P-hacking, or where the dataset is unusably small and so on and so forth.

Hence why I find myself agreeing with the other posters - if you find yourself strongly disagreeing with Jeff Bezos about something that seems to be within his domain, think twice.

The statement is saying that in many situations, data is improperly or incompletely gathered, analyzed, and/or applied. Refusing to attempt to understand why an intensely data-driven individual would make that remark doesn’t seem wise. :)

> the vast majority of the time it is simply a combination of selection bias and the human tendency to want to believe things that support your own worldview

Citation needed.

Given Jeff Bezos success in growing a business from scratch, this is clearly one of his secrets and why "Are right, A Lot" is an Amazon leadership principle. I'm always surprised that people are so easy to dismiss insights from successful people.

> Given Jeff Bezos success in growing a business from scratch…

One thing we have to remember, savant level of greatness in one area can often indicate deficiencies in other areas.

If I were starting an online retailer 20 years ago, I would absolutely take Bezos’ advice. There are also countless other areas where I would seek advice from those with different areas of experience.

Steve Jobs was fantastically successful at running a computer company. He's primarily dead because his business sensibilities were astute and on point while his knowledge of heath care was so laughable that one of the worlds wealthiest men ignored reasonable treatments that had a reasonable chance of saving him so long that all the money in the world couldn't. It seems that skill in business does not confer virtue in any other area of life and one ought not give a very successful person extraordinary consideration outside his area of expertise.

You have been presented with a thesis that the California exodus is a myth. You have responded with a complete non sequitur quote from a mediocre man about trusting the anecdote vs the data asking us to forget or ignore in order the fact that the idea espoused by the quotation is nonsense, that Bezos wasn't registering an opinion on this topic, you are, and that you have made no argument whatsoever.

It is untenable to launder your opinion through a rich dudes mouth and then accuse skeptics of casual dismissal of insights from successful people when people are actually and in fact not dismissing Bezos they are registering their opinion on your comment here today.

Just because Bezos knew how to create a successful business, does not make him a successful person.

The election wins of Trump, Boris Johnson, Brexit and The Liberals in Australia show that data collection really matters. Similarly the financial crisis of 2009 shows that missing data can really matter.

That's the key point of the quote:

> There's something wrong with the way you are measuring it

But those are all examples of data collection working fine, though, and people not listening to the data because their anecdotes went the wrong way.

FiveThirtyEight predicted a 28.6% chance of a Trump victory: https://projects.fivethirtyeight.com/2016-election-forecast/ This is a number far greater than zero - it is, for comparison, more likely than flipping two coins and them both coming up heads, or drawing a random card from a deck and having it be diamonds.

Opinion polling of the 2019 Conservative Party leadership election showed Boris Johnson favored to win: https://en.wikipedia.org/wiki/Opinion_polling_for_the_2019_C...

Opinion polling on Brexit showed that it clearly could have gone either way, with a narrow margin between the two options: https://en.wikipedia.org/wiki/Opinion_polling_for_the_United...

I'm less familiar with Australian politics, but assuming you mean the 2013 election, opinion polls showed the Liberals were clearly in the lead: https://en.wikipedia.org/wiki/Opinion_polling_for_the_2013_A...

If people look at data that says "28.6% chance of winning" and they choose to read it as "0% chance of winning," that's not a problem with the way anyone's measuring anything, that's a problem with people trying to treat data as if it's an anecdote.

I agree it's a bad example re: the Bezos quote, but I also don't think it's an example of anecdotes leading people astray. IMO it's from the brand of bad pop sci that misinterprets perfectly good data while ranting about how amazing science is and how stupid anecdotes are.

> I'm less familiar with Australian politics, but assuming you mean the 2013 election

They definitely mean the 2019 election.

> FiveThirtyEight predicted a 28.6% chance of a Trump victory:

This is cherry-picking of the highest widely-known prediction by far, practically an outlier. And I don't think they do any actual polling, just analysis using others' polling data.

I hate to call something gaslighting, but this is gaslighting of the highest order. I am not familiar with Australian politics, but the other three cases were textbook cases of the data saying one thing (Trump might win, BoJo will probably win, and Brexit might happen) and people refusing to believe it.

How these get turned to make precisely the opposite point (Not just by you. It happens all the time) never ceases to amaze me. Presumably the thought process is "I and the people I respect build beliefs based on data. I and the people I respect were wrong. Therefore the data was wrong". Unbelievable.

What exactly is “the data” that we’re talking about? Is it the measurements that were made, and the methodology used to make them? Or is it the conclusion that something in this case is a “myth”? Because I’d suggest that only one of those things is “the data”, and the other one is a subjective interpretation of it. I’d also suggest that people who have an agenda to promote will often attempt to blur the line as much as possible between empirical measurements, and their own opinions about what they’re supposed to mean.

In this case, the data I am talking about are the polls. Of course, there's a lot of methodology that goes into designing the polls and the data collection process, but those were basically solid and importantly, not subjective.

> I’d also suggest that people who have an agenda to promote will often attempt to blur the line as much as possible between empirical measurements, and their own opinions about what they’re supposed to mean.

This ran rampant in all those cases (Trump, BoJo, Brexit), I agree.

There was data and then there was the NYT saying Hillary has a 99% chance. Ha

The Australian election results were within the margin of error of the polls.

https://www.theguardian.com/australia-news/2019/may/19/voter...