During COVID there were multiple cases like this where a study got a lot of hype and discussion from non-experts and turned into "the science says X", when in fact the science was as of yet extremely unsettled. Sure enough, as the experts came to a consensus it rarely matched the public's initial perception, which led to a lot of confusion, conspiracy theories, and fingerpointing.

Science-as-spectator-sport is fun, but I worry about the impact it will have on society as a whole and on the execution of science in particular. How many research decisions will be influenced by the possibility of going viral? How many bad decisions will be made as a result of pressure from millions of non-experts who briefly become armchair X-ologists?

When science is done badly (which arguably shouldn't be considered science), which then leads the public to have elevated expectations, only then for science to be done right and disprove and reject the original "findings", public trust in science is ever so slightly damaged.

PR is something we cannot control -- and avoiding releasing results in a (semi-vain?) attempt to control PR arguably does more harm than releasing them does.

The sea makes waves as it will. We can moderate as much as we are able. I think the rest is simply a matter of accepting that things happen as they do out of our control. We can only truly impact ---- and even then, not necessarily guaranteed! D: ---- -- in some potentially very small part -- whatever sphere is around us, and I feel that that's a collective individual responsibility.

Journalists are going to write nonsense, hype-filled science articles. PR flacks are going to hype puff newswire blurbs. Why? Because that's what they're paid to do.

You can curse the existence of bad incentives, if you want annoying ideologues to call you a communist.

Or you can hire your own PR flacks. Because worrying about how people will react to what you're doing is PR, and going up against professionals without your own is like going to court without a lawyer.

Or you can just, you know, do science, accept that people suck sometimes, and get on with your life.

Having had some work get incredible attention, and other work not at all, I've experienced a small slice of the volatility of the web. My most tweet chain was a semi-technical vent I wrote over the course of 30 minutes after stewing about some semi-useless technical hype that no one seemed to be addressing the flaws in. I wrote it in a way that was more attention-grabbing somewhat than my more technical posts, put it out there, and shared it in a few places.

Two days later or so, my number of Twitter followers had over quadrupled.

I think humanity can be quite finicky sometimes (a more general statement, I don't think one could conclude that from the previous anecdote alone).

This is just another example of the media’s negative effect on society. Similarly, the media endlessly poring over every detail of the Ukraine war has likely made their job much more difficult because it damages the element of surprise.

The media originally began as something quite negative with what we called “yellow journalism.” Then for a century or so we saw a kind of golden age of journalism where newspapers had strong reputations to uphold but were fairly rewarded for it through ads and classifieds.

Now the media is back to yellow journalism (clickbait) and eroding the institutions of society.

How many worse decisions are made by people who can't read and won't learn about the nuances of a topic?

Ask L Ron Hubbard?

The willingness of the crowd to believe in counterfactual things is not constrained to science, and whatever damage/risk is posed by that is not new - as Galileo can attest.

What the general public does not see it the regular flood of papers that pretend to change the world and that turn out to be bogus. So, from an insider point of view, the issue is that what we are supposed to avoid (crack pot theories becoming mainstream or getting too much traction) happened in a spectacular fashion. So a lot of people get excited about nothing and then end up distrusting the scientific process itself (“they don’t know what they’re doing”, “they make everything up”, “they write a lot of nonsense”, etc).

In this case, I think it turned out to be a good thing. People got excited, some of them thought about possible implication, others managed to pick up some notions of material science. The enthusiasm and activity from people trying to replicate and investigate the material was heart-warming. But yeah, it was bound to finish like that.

> Why would I ever be inclined to feel that there's pie on my face now that we've got fairly strong evidence refuting the claims?

You really, really don’t want to be seen as a crack pot when your funding and career depend on how external people evaluate your work. You also really, really don’t want to have to retract a paper because you’ve missed something obvious. Retraction is a traumatic process even if you are in good faith. This is sidestepped by releasing preprints (so no peer review and no risk of retraction). But at the same time this is a reason why outlandish preprints tend not to be taken too seriously. There is less incentives to get it right.

This should be taken in the context of room temperature superconductors being notorious physics vaporware along with practically useful advances in quantum computers and useful fusion. What these have in common is a sort of holy grail status, where it's obvious they'd be a revolutionary complete game changer. Not that any of these things are obviously impossible, there's just been so many instances of discoveries in these areas that have failed to replicate that there's inevitably a lot of eye rolling in physics when these types of findings are announced.

This is probably because people (i) were not aware that there had been many other hypes about RTSC before but less publicly visible all proved to be false, (ii) not being able to accurately judge the technical quality of the initial evidence, (iii) uncritically believing that the data in the initial preprints was proof for superconductivity because their authors said so.

While a convenient abstraction, it plays into our biases to notice and remember only some of the discourse.

Plus I don't think it's really relevant to what I'm saying given I'm not making a claim about how any specific individual or group reacted, but that it's odd when there's people who treat an optimistic outlook as an error.

The last few weeks with the LK-99 hype combined with the usual ChatGPT stories, I actually started feeling that maybe the site should be renamed Hype News.

> but that it's odd when there's people who treat an optimistic outlook as an error.

IMHO it’s best to treat any extraordinary claim as BS until proven otherwise as it’s very easy to concoct BS claims. If we take every one of them seriously, it will consume all of our attention and destroy the signal (actual facts) to noise (unproven claims) ratio on this site.

An optimistic outlook without a semi-plausible basis that you can convincingly elaborate on, or link a vaguely credible source doing so, IS an error, at least going by HN norms.

It is just an attitude that values positive possibilities over fretting about negative possibilities.

Especially in cases where there is a small chance of a huge upside, relative to virtually no downside. We didn’t lose any superconductors. :)

I don’t recall anyone on HN erroneously declaring the material was definitely a new superconductor before subsequent evidence arrived at a consistent conclusion.

There is nothing wrong with optimism.

To clarify, I was referring to "An optimistic outlook" in terms of actual assertions/claims/etc. that are written down on-the-record in public.

Of course HN users can have the general abstract sentiment of optimism at anytime in their mind. I don't think there are any norms around internal sentiments.

For the scientists getting their hands on a breakthrough, the risk and reward was worth it, but for the public at large? No one should care until there are definite results.

It's pretty standard to be skeptical of extraordinary, poorly supported scientific claims and you didn't have to be an expert to find out experts were fairly skeptical of this from the beginning and the reasons for their skepticism. The broad HN sentiment was at odds with what you could find elsewhere. This isn't a moral failing or anything, just a common mode of HN-like forums but to elevate it to some some sort of positive rather than a thing to be cautious about seems backwards.

What a shamefully foolish intellectual failure!

Mostly I saw actual physicists who had experience in the field being very skeptical, throwing a lot of cold water on the fire, and pointing out that the original authors looked like amateurs.

And then I saw a lot of people with zero experience in the field running around yelling about how they were out of touch, how this was a revolutionary new way that science would progress on twitter, out in the open, etc. People who were skeptical got called all kinds of names.

It didn't help that a lot of people on twitter pivoted from crypto-hype to AI-hype to LK99-hype pretty much on a dime.

There was also a lot of highly upvoted comments with the usual thoughtleadering style of "let me beak it down for your, here's the ELI5 of what is going on an what the implications will be..." followed by whatever they learned in the past 48 hours from plowing through wikipedia articles.

There could be a lesson here about listening very carefully to experts in the field when they give you their opinions. They often sound very highly biased, but there's usually very good reason for that. Once in a lifetime there's the event where some paradigm is overthrown and all the old scientists look a bit foolish because their instincts were to be skeptical -- but those instincts came through a lifetime of correctly being skeptical 999 times out of 1000 about wild claims in their field.

This could be a teachable moment that could inform people about climate change, coronavirus and other scientific claims. If you want to disagree with experts in the field you really need to get off your ass, get off twitter and the blogs, and go do the hard work of understanding what the scientists actually know by reading the articles that they publish. They're very often correct and their opinions hold more weight because they've literally spent their lifetime learning and thinking about this one thing. They didn't start learning about superconductivity / viruses / climate last week and you need to do better than some showerthought or wishful thinking that you think proves your viewpoint.

But we're not going to do that because its only been a few days and we've literally forgotten about how much flak scientists were getting on here over skepticism towards the initial claims.

And I had some of the most positively stupid arguments on here where people were trying to assert that scientific experts needed to express exactly zero bias because they were experts and held to a higher standard than the average moron with no experience who could argue whatever they liked. Engineering a rationale to be able to reject anyone with a strong opinion based on expertise in favor of strong opinions from randos on twitter.

I addressed why.

One of the things I look forward to in an HN discussion is the comments of people who can collate expert opinions on the subject and surface these kinds of points in a more ELI15 kind of way. But despite there being ~a week of LK-99 stories permanently on the front page, there wasn't much of that (a little on the initial thread, and virtually nothing for the next several days)--and it's not for lack of physicists commenting on the topic (in other forums)!

[1] I saw someone point out that, when you translate the units on the resistivity/temperature graph, it is a worse conductor than copper at room temperature, below its claimed critical temperature.

HN is full of subject matter experts on computing-- that is, software, and to a lesser extent, hardware-- beyond that it's a mixed bag at best. Even as an interface designer, I see so much confidently presented and totally bogus pseudo-expertise on art and design here that it's actually kind of funny, and that's much more closely related to software development than physics is. That BS sounds credible to other developers because it's in a developer's voice and trips on misconceptions common among developers. I suspect that's true with the other non-computing topics discussed here that I don't know enough about to give an expert opinion on.

As a long-time developer myself, I've been on both sides of assuming our astonishing intelligence and analytical capability can make up for lacking the requisite expertise. The mistake is expecting the HN crowd's musings about things outside of it's expertise to be more trustworthy than any other internet forum. If this were some physics subreddit or something like that, the criticism would make more sense. This is just people being excited by something a lot of other people were excited by.

I can tell you from first hand experience, much of the time subject matter experts are often downvoted into oblivion by the HN hive mind. To the point where you only see clueless people at the top.

Happens to me regularly when it comes to machine learning, neuroscience, education/university threads.

For example, people say crazy things about things like university admissions or grad student salaries. Never mind about ML where most of the information here is just wrong.

That’s why many started with dimagnetism indeed.

That’s definitely not true despite your attempts to gaslight us.

There was a dose of Dunning-Kruger, and some software engineers telling us how to science, but there was also a lot of engagement and interesting discussions with genuine experts. Overall I found it quite interesting to follow.

> uncritically believing that the data in the initial preprints was proof for superconductivity because their authors said so.

It had undertones of small team (in a private institution, no less) taking on the stodgy establishment, which is quite popular among some people here. The concepts are also not very difficult to grasp on surface, so a lot of people can form an opinion, however well founded.

People complain about peer review and scientific publisher as well. It is not difficult to see how this could push them to champion something that comes from arxiv.

Kinda buried the lede there. A lot of folks around here and in my own orbit were practicing the former while excluding the latter, or worse, were taking shots at people trying to inject some level of rationality into the conversation. Heck, some folks even went so far as to refer to those types of counterpoints/comments as just a "bizarre reaction"...

rock music

I feel like this line is doing a lot of lifting in your comment.

The problem is that as lay people we are completely unequipped to gauge a claim like this. I followed along on HN and there were plenty of posts by people who were giving LK-99 crazy odds of success, fueled in no small part by viral videos of outright hoaxes from pseudonymous "researchers."

It's fun, in a science fiction-y way, to speculate on what a material with the supposed properties might have meant for the world, but the degree of skepticism that should have been applied was lacking for many.

There's a tendency on HN and similar forums to devour new developments - almost a fanaticism about learning the newest/latest/best before the general public. But in this case, a truly extraordinary claim had been proposed, and it was even published without the researchers' consent. There was precious little reason to give it any attention at all at that phase.

If people had viewed LK-99's properties as "almost surely science fiction" all along, I could find myself agreeing with you, but that's really not how this played out. Sadly this event showed there's a market for hyping up weak claims that people will be poor at evaluating, and I guess we can probably expect more of them.

I don't see it as in any way an unique event, and also not unique for the enthusiasm seen on this site. The "believers" in most of the hypes typically aren't cured fast, as the article notes:

"While some commentators have pointed to the LK-99 saga as a model for reproducibility in science, others say that it’s an unusually swift resolution of a high-profile puzzle. "Often these things die this very slow death...""

I think the public reaction in this case is a symptom of a problem with our information ecosystem that extends beyond science. Just because something is fun to participate in in the moment doesn't mean it's not harmful to the underlying scientific/political/social process.

A significant reduction in room temperature resistance would still be incredible, even if it wasn't a "room temperature superconductor." Might still enable a lot of those "exciting conversations." Just not some binary yes/no computer holy grail.

Also, big effect was scientists went "Whoa. There's a whole mode/regime of resistance change we never really looked at." The modeling papers that came out almost immediately were really interesting. Might still have cool applications.