News for hackers. Stuff that matters.
Re:Are Experts Real? by DanielBMarkham on Monday January 11, 2021 @12:21PM (#25732179)
This essay points out a pervasive problem that doesn't get much attention (aside from fear-mongering)

Because it's a topic so easily prone to hyperbole, it helps if we can reduce it down to a few premises.

- Science is science, but the practice of science is a social system. People get tenures, jobs, promoted, invitation to speak, etc. by how well they do in the judgment of their peers

- Political systems exist for political reasons, ie, if you don't have a reproducible, falsifiable test in a feedback loop somewhere, your career and goals are all about image. There's no alternative, either for you or your field of study (That doesn't mean other sciences aren't important or aren't really science.)

- If somebody appears on TV or in the media playing the role of an expert, it is, by definition, because they wanted to appear on TV or in the media. The goals of being popular and asked about important questions and the goals of whatever your field are ... two different things. The reporter doesn't care who figured out X. They care who can bring excitement and emotion to talking about X.

- All this leads directly to the problem of the average science news consumer: any communication you receive about science or research is the result of a lot of conflicting interests. "Being good science" is way down the list, sadly. I imagine the percentage of average readers who understand p-hacking is below 5%

- None of this is a secret. The average news consumer knows it, even if they're not able to articulate it

- Add in things like the reproducibility crisis, and we've got a huge problem as a species. We've got one flashlight to light our way in the darkness and it turns out there's a loose connection in there somewhere

Re:Are Experts Real? by ska on Monday January 11, 2021 @12:54PM (#25732819)
> ...by definition, because they wanted to appear on TV or in the media

I suspect this misses some nuance; there is a meaningful difference between people who appear on the media because they want exposure, and people who do it because they want their field of study to be well represented.

For example there are people to accept interviews when they genuinely dislike the process, because they felt it was a duty that came along with their chair, etc. I doubt these are ever the most dynamic interviews, but they are common.

I agree this doesn't really extend to the level of the same old bobble heads pulled up on prime time news TV, but don't thing you can paint all of the media exposure with this brush.

Re:Are Experts Real? by grawprog on Monday January 11, 2021 @01:15PM (#25733198)
I think that comes down to self-proclaimed experts vs experts as decided by their peers.

Many of the people I met who were considered experts and had a vast depth of knowledge never referred to themselves as experts and seemed do get embarrassed at the idea.

Whereas, i've met several self-proclaimed experts who liked to talk a lot about how great they were, but never really showed the depths of knowledge the other kind of experts showed.

At least from my experiences anyway in life.

Re:Are Experts Real? by potta_coffee on Monday January 11, 2021 @02:45PM (#25735237)
Peer incentives and desires can also be skewed / wrong. Just because there's some kind of consensus doesn't mean the conclusion is correct.
Re:Are Experts Real? by mc32 on Monday January 11, 2021 @10:20PM (#25741837)
Is it better to have the right framework but come to the wrong conclusion (for now), or is it better to reach the true conclusion with the wrong framework or basis? I guess it depends on timeline.
Re:Are Experts Real? by pdonis on Tuesday January 12, 2021 @01:45AM (#25743265)
> I think that comes down to self-proclaimed experts vs experts as decided by their peers.

I think the correct criterion is neither of these: it's whether the expert can back up their expertise with models that have a good predictive track record. That's the real test of whether they actually know what they are talking about vs. just blowing smoke. Unfortunately, this criterion is hardly ever applied that I can see.

Re:Are Experts Real? by sigotirandolas on Monday January 11, 2021 @01:12PM (#25733148)
I see it as a spectrum, where to be an "expert" you need to both skilled in the field and a good communicator/politician.

The problem is, in those areas where there's not a tight feedback loop as mentioned in the article, it's possible to build your way to the top by just being somewhat skilful in the field and knowing how to appear confident, and how to (over)sell your work, and knowing when something is technically right but you should not say it because politics. And on your way to the top you're going to out-compete those that are very skilful on the field, but don't have such good communication abilities.

Re:Are Experts Real? by Timpy on Tuesday January 12, 2021 @07:06AM (#25745758)
I agree with this. Let's say for example you are one of the world's leading immunologists when a global pandemic breaks out. Going on TV because you "by definition want to appear on TV" is tautological, you could say that about anything on the list. A leading expert may see that they have a responsibility help lead the public, it's perfectly possible that want for media attention for personal fame is completely irrelevant. Want for media attention to help guide the course of a pandemic to a safer path qualifies as "because they wanted to appear on TV," but the GP reads uncharitably.
Re:Are Experts Real? by tpoacher on Monday January 11, 2021 @01:52PM (#25734121)
I don't see how 'disliking the process' equates to 'not benefitting from the outcome'.

I dislike interviews, but I like having a job.

Re:Are Experts Real? by kurthr on Monday January 11, 2021 @03:10PM (#25735760)
Well, since many of these "experts" are professors with tenure they can't really lose their job nor readily increase their salary. They might be able to more easily get funding for their research/students from nontraditional sources (e.g. industry grants), but many of them would not, because they value the estimation of their peers which might be (rightfully) hurt. Not all or even a majority even care about money or they wouldn't have chosen the profession. Some of this has changed with startup board positions, industry coalition funding, and academic promotional departments, but not quickly.

All of that goes out the door with pundits and shills, who don't just appear once, but have chosen the field for fame/fortune... and for those who have largely chosen to sell their research topics/results for profit. However, those are relatively easy to spot by most viewers and journalists who care.

Re:Are Experts Real? by ska on Monday January 11, 2021 @03:43PM (#25736454)
There isn't any direct benefit for many of the people I had in mind.

Indirect benefits are harder to evaluate and will be nature involve a bunch of hand waving, but there is a huge difference between say holding a research chair, and having a consulting business as an "available expert".

Re:Are Experts Real? by DubiousPusher on Monday January 11, 2021 @02:23PM (#25734793)
> I suspect this misses some nuance; there is a meaningful difference between people who appear on the media because they want exposure, and people who do it because they want their field of study to be well represented.

I don't think there is a meaningful difference. First, I don't believe people can make choices in any kind of isolation. If you believe you are making a choice without your ego involved, you have simply deceived yourself. Congrats, you have a high functioning ego. Second, even if they could, audiences are not capable of concluding with any reasonable certainty which is motivating someone appearing on television. I don't mean that derisively, like people are stupid, I mean it is essentially beyond humans to do this.

IMO, in the absence of actionable information, it is pointless to speculate. Basically, you cannot know it so it's not worth thinking about.

Re:Are Experts Real? by ska on Monday January 11, 2021 @03:53PM (#25736676)
I think I see your point - from the point of the viewer it's impossible to tell why someone has agreed to be interviewed.

I guess my hypothesis is that you get better information from people whose reason for agreeing to interviews is not self promotion. If true, that's actionable.

Re:Are Experts Real? by coldtea on Tuesday January 12, 2021 @12:11AM (#25742577)
>- Science is science, but the practice of science is a social system

Well, there's no science - except if one is a Platonist (and believes in the concrete, and even real-er than real- existence of ideas).

There's just the practice of science.

And even if there was a science outside of its embodiment in practice, what would it be? Where are its iron laws written? There are 100,000 books on epistemology, the philosophy of science, methodology, and so on, but there's no concrete singular set of guidelines or tenets of practioners to uphold at every point (besides generalities like "experiment", "falsifiability", "process" etc).

And these things mean even less so in soft-sciences, from sociology to economics.

>The goals of being popular and asked about important questions and the goals of whatever your field are ... two different things.

It's not just the media goals that are at odds with your "pure" field's goals.

So are (or can be) the goals of your university administration, of your state, of your industry sponsors, of your vanity and/or greed, or your career advancement, of those that give grants, of the public opinion, of your personal ideologies, and so on...

Re:Are Experts Real? by pdonis on Tuesday January 12, 2021 @01:43AM (#25743246)
> Well, there's no science - except if one is a Platonist (and believes in the concrete, and even real-er than real- existence of ideas).

There's just the practice of science.

I disagree.

There are scientific models that make predictions and can be judged by the accuracy of those predictions. That is true regardless of how those models came to be--who built them, what credentials they had, how their peers regarded them, or even what philosophical beliefs they had (Platonist or otherwise). The models and their predictions, and how accurate (or inaccurate) those predictions are, are all independent of all that. Those models are science separate from how science is practiced as a human social endeavor.

So I would say the answer to the article's title question is simple: If the experts are basing their expertise on models with a good predictive track record, then yes; otherwise no.

Re:Are Experts Real? by coldtea on Tuesday January 12, 2021 @02:24AM (#25743485)
>There are scientific models that make predictions and can be judged by the accuracy of those predictions.

That's neither here nor there though.

Those models might be abstract (in the sense that they're not concrete object) but they are still an example of concrete practice of science -- they were thought and formulated into specific words, measurements and papers.

In other words, models come to life by the concrete practice of science by humans (with all its faults). They don't manifest by some abstract science.

>Those models are science separate from how science is practiced as a human social endeavor.

It's the opposite.

Those models were built not "separate from how science is practiced as a human social endeavor" but exactly by people practicing science as a human social endeavor. And they were shaped by exactly the things you say they were not: who built them, what credentials they had, their beliefs and imperfections (e.g. lack of statistics knowledge), etc.

For example, if the scientists (actual persons) creating a model is greedy, they can create a model that advances their financials (rather than science) by optimizing for grants or publicity or "fancy results" or sponsors, against accuracy.

One might still agree, sure, but the model can still be verified independently of those bias and personal faults etc.

But that would be wrong too. Because there's no abstract verification. Verification also happens by actual people, as an actual process, and prone to the same issues: slopiness, greed, shallow peers merely linking to a paper they don't understand well, ideological biases, simply not caring, and so on.

They're not the abstract idea of science.

>If the experts are basing their expertise on models with a good predictive track record

Even that is not some fool-proof criterium.

Because you can always either make the model vague enough or shape, manipulate, skew, explain away the subsequent measurements enough to get a "good predictive track record". It's much easier in science than it is in sports bets where a team either wins or doesn't.

See, the verification of those predictions also happens by humans (with all the above issues).

Re:Are Experts Real? by pdonis on Tuesday January 12, 2021 @02:48PM (#25752429)
> That's neither here nor there though.

I disagree. I understand the issues you're raising, but there is a key difference as far as we, members of the public looking at science, are concerned. The process of building models can't really be critiqued by non-scientists; but the process of verifying the predictions of models can, because predictions, at least predictions that non-scientists care about, can always be boiled down to something that anyone can test. For example, if you want to test whether relativity works, just use the GPS on your phone: if the location it tells you you're at is correct, you've just verified predictions made by relativity.

It is true that scientists do not always boil down their predictions to such simple tests, but as members of the public, our response to that is easy as well: if a scientist can't boil down the predictions of their model to a simple test like that, don't believe the model. In other words, we as members of the public can give incentives with regard to predictions that we can't give with regard to model building, or more generally scientific research as a whole.

> you can always either make the model vague enough or shape, manipulate, skew, explain away the subsequent measurements enough to get a "good predictive track record".

You can only fake tests that aren't real tests anyway. You can't fake something like GPS working: either it consistently tells you where you are correctly, or it doesn't. So that's the kind of tests that should be required before members of the public accept the claims of any scientific model.

Re:Are Experts Real? by ernst_klim on Tuesday January 12, 2021 @04:39AM (#25744435)
> There's just the practice of science.

Sure. But author doesn't state that there is ideal science (STEM) and science muddled with politics (social sciences).

Author says that there are good social institutions (like American judicial systems) and bad ones (like Russian judicial system), and that STEM is the case of the former one (although it has quite a lot of problems as well), while social sciences are rather a case of bad corrupted institution.

I think that author's idea of feedback loop is really interesting and reminds me of how things are going in other social institutions as well.

When we have feedback loops which incentives corruption, we get corrupted badly functioning institutions and vice versa. I think ``Why nations fail'' approach this from the same angle.

Re:Are Experts Real? by angrais on Tuesday January 12, 2021 @04:46AM (#25744482)
How you've described the social sciences being mudded with politics is the same in large parts of STEM, particularly the Technology and Education parts. Of course, as GP stated, everything is political so securing tenure, being invited to speak, etc, are political and social decision making processes that impact all aspects of STEM.
Re:Are Experts Real? by ernst_klim on Tuesday January 12, 2021 @04:54AM (#25744565)
> How you've described the social sciences being mudded with politics is the same in large parts of STEM

Sure, as anything else. And I've explicitly stated that this is not the point.

The point is that politics leads to bad science in former case and good (relatively and for the most part) science in case of STEM, exactly like politics leads to great judicial system in US and terrible judicial system in Russia.

Because politics could lead to great checks and balances and good incentives, or it could lead to cronyism, corruption and regulatory capture, based on how the institution is constructed/designed, and its relations with other institutions.

Re:Are Experts Real? by Nydhal on Monday January 11, 2021 @10:18PM (#25741826)
> The average news consumer knows it, even if they're not able to articulate it.

Are you sure about this? Did you spend a lot of time interacting with average news consumers or is this based on other data?

Re:Are Experts Real? by DaiPlusPlus on Monday January 11, 2021 @11:21PM (#25742245)
What even is an “average news consumer”?
Re:Are Experts Real? by cbozeman on Monday January 11, 2021 @12:39PM (#25732528)
This is a fantastic comment, and spot on I think. I concur with everything you've said.
Re:Are Experts Real? by dboreham on Monday January 11, 2021 @09:15PM (#25741406)
There's an implicit assumption here that someone who refuses to appear on TV when asked has a more valuable opinion than someone who accepts. This seems on its face to be false.
Re:Are Experts Real? by s5300 on Monday January 11, 2021 @09:59PM (#25741703)
While I mostly agree with everything you've said

I truly believe

>...The average news consumer knows it, even if they're not able to articulate it

Is very far from the truth.

Re:Are Experts Real? by breck on Monday January 11, 2021 @01:08PM (#25733065)
The solution is to #abolishcopyright. If you don't allow people to exchange ideas freely, and you allow entities to control monopolies on distributing information, you are going to have a vastly suboptimal flow of information.
Re:Are Experts Real? by bergstromm466 on Monday January 11, 2021 @05:04PM (#25738029)
> The solution is to #abolishcopyright.

That's the spirit! And abolish all privatized and monopolized knwoledge. Abolish Silicon Valley.

"Place Silicon Valley in its proper historical context and you see that, despite its mythology, it’s far from unique. Rather, it fits into a pattern of rapid technological change which has shaped recent centuries. In this case, advances in information technology have unleashed a wave of new capabilities. Just as the internal combustion engine and the growth of the railroads created Rockefeller, and the telecommunications boom created AT&T, this breakthrough enabled a few well-placed corporations to reap the rewards. By capitalising on network effects, early mover advantage, and near-zero marginal costs of production, they have positioned themselves as gateways to information, giving them the power to extract rent from every transaction.

Undergirding this state of affairs is a set of intellectual property rights explicitly designed to favour corporations. This system — the flip side of globalisation — is propagated by various trade agreements and global institutions at the behest of the nation states who benefit from it the most. It’s no accident that Silicon Valley is a uniquely American phenomenon; not only does it owe its success to the United States’ exceptionally high defence spending — the source of its research funding and foundational technological breakthroughs — that very military might is itself what implicitly secures the intellectual property regime.

Seen in that light, tech’s recent development begins to look rather different. Far from launching a new era of global prosperity, it has facilitated the further concentration of wealth and power. By virtue of their position as digital middlemen, Silicon Valley companies are able to extract vast amounts of capital from all over the world. The most salient example is Apple: recently crowned the world’s most valuable company, Apple rakes in enormous quarterly profits even as the Chinese workers who actually assemble its products are driven to suicide." [1]

[1] Wendy Liu, https://tribunemag.co.uk/2019/01/abolish-silicon-valley

Re:Are Experts Real? by rramadass on Tuesday January 12, 2021 @12:22AM (#25742648)
Great article; thanks for the link.
Re:Are Experts Real? by ernst_klim on Tuesday January 12, 2021 @05:02AM (#25744641)
> able to extract vast amounts of capital from all over the world

Wow. Because bringing workplaces to China made both Chinese and Americans worse off, you know? It would be much better if Chinese had continued work on rice fields and Americans would buy electronics assembled in US at much higher prices. To hell this globalization.

(sarcasm)

Such a "economics is a zero sum game mindset", ridiculous.