When I was young, lonely and isolated, the internet was a godsend: it connected me to like minded individuals and expanded my horizons. It exposed me to a world of ideas about science, technology, history, politics, music, etc. at a time when everyone around me was mentally stagnant.

That same power to connect can lead people down a dark or dangerous path. Approach the internet with an uncritical mind and you could emerge as a flat-earther, anti-vaccer, or worse.

I've tried to explain to a friend why vaccinations are a good idea, and I get a bunch of crap about how modern medicine doesn't work for her, and how she took a homeopathic vaccine, went to India, and didn't get ill.

I once knew someone who bought an expensive electronic detoxification device. The idea was you held on to the handles and it sent purifying waves through your body, killing any parasites. She insisted on showing off her stool after the process was complete. She said she could see the little "bugs" that had been in her system. Then we discovered that she had forgotten to put in the batteries.

I knew someone else who wanted to invest in a $50,000 machine that made similarly dubious claims. The idea was to make the money back by selling treatments. Thankfully her daughter talked her out of it.

Gullibility is a dangerous thing, especially at the intersection of personal health and second party financial incentive.

Normal, grounded people don't wake up one morning and go "I get it, the Jews and Soros are conspiring to suppress the truth about the Flat Earth", they get there by baby steps, each one veering more and more into the illogical.

Maybe it was when they were unemployed or in medical trouble and stressed out beyond their ability to cope, then wanted explanations, however implausible, that gave them at least an idea of what was wrong with their world. From there it took root and by degrees they slip into this other world of nonsense and batshittery.

Crystal therapy and detox can lead to healing rituals can lead to cults can lead to suddenly selling everything and donating it all to some fraudulent healer.

Like you observe, I think we're all in need of a best friend, family member, or spouse who can say "You've got a problem, this has to stop now."

The problem with the internet is it delivers. If you want to know more about quantum physics or molecular biology or cancer research you can dig and dig and keep on digging, you will never run out of material. Likewise if you're looking for harmful material, it will unquestioningly keep going all the way.

If Google started to detect the pattern of someone slipping into madness and did what they could to start to refute things, to sprinkle in some helpful counter-points with their diet of pure nonsense, maybe we can save people who'd otherwise be lost.

To which the response is who are Google to influence how or what people think?

It's easy to make up pure-nonsense examples that nobody believes - like Soros suppressing the truth about the flat Earth - and say believers should be saved.

What if someone questions the official version of 9/11? What if they believe/d the NSA taps internet, social networks and world leaders? What if they think an omnipotent sky fairy watches over us all and blesses America?

Google's "did you mean" is irritating enough when applied to assumed typos; its purpose should be to give people what they're looking for, not tell people what they should be looking for instead.

But they are already doing just that. I watched a few Youtube videos related to Quantum Mechanics and my recommended queue was flooded with videos full of pseudo scientific nonsense.

It took me considerable effort in having these weird videos no longer recommended to me. I think we can just be honest about what caused it, it's a lack of intelligence in the sorting algorithm, that currently isn't able to distinguish between true and false statements.

I understand there are areas where this line is vague, but if I watch a few videos about astronomical events, why am I getting recommendations about aliens visiting earth?

Give people a long leash, but when they're way off to the end of the bell-curve of normality throw them a life-line.

> It's easy to make up pure-nonsense examples that nobody believes...

That's actually an example of things people do believe. If you engage with some of those wild outliers you can learn a lot, quickly. Some of them are trolling, but many of them have extremely firm convictions. Maybe it's Sandy Hook. Maybe it's chemtrails. It's usually a bunch of stuff, all tangled together.

Hello User. We've noticed you've been spending more time on StackOverflow than 99% of our other users. May we suggest you spend some time looking at cat videos to balance things out? -- Your friends at Google

It's like I usually say

> You can't fix crazy and you can't fix stupid.

Maybe. Or, perhaps it's the tendency of irrationality to beg more irrationality today, when a large number of vectors exists in which the irrationality can spread. If someone said something irrational and blaming 30 years ago, a few people might hear it and think, "huh, that's a weird thought". Most would probably not repeat the thought to anyone later because most people don't want to be viewed as irrational or weird by others. Maybe a few people might, because they were already irrational, but the chances were low that enough would repeat it to where it could spread.

30 years later, two things have happened to increase the likelihood of a "viral irrationality" to root and grow at increasing rates: 1. lots of people on the Internet and 2. lots of irrational things being said on the Internet. I would classify most of the irrationality on the Internet as others speaking for others thoughts or feelings. Check out Alex Jones just pulling shit out of his ass for example, under the guise of "news", but really just being a performance artist. He's "entertaining" people by pretending to be trustworthy, and keeping it going by making anyone listen to him question the "real" news (of which is he is not a member) to keep the schtick going.

I look at it more like a weird NP problem, where increasing amounts of time are needed to unravel the logic being presented as it spreads further into the aggregate, given it's self-referential in nature to answer leading questions with more irrationality. The blame game people play might be likened to two traveling salesmen trying to solve each other's problems of shortest path while also traveling paths that avoid or overlap each other. Given certain paths, the salesman may find themselves in a loop, blaming each other for being stuck, but unable to stop doing the calculations to get themselves out of the pickle.

Check out the concept of a double bind. It's relevant to this discussion.

There's so much wrong with this...

Reminds me of the audiophile that was very happy with his top of the line amplifier that he custom built. It was clearly much better than the previous one.

Turns out he had forgotten to turn it on and was using the old one.

The Internet is perhaps the invention that most closely parallels the invention of the printing press. But, unfortunately the Internet appears to be working paradoxically to the development of knowledge.

Is the age of the internet the harbinger of an age of disinformation, alternative facts, and ignorance?

Is it possible that the Internet could cause our cognitive devolution as a species?

https://jimdroberts.wordpress.com/2017/04/23/the-information...

This happened already in the early 2000s. For over a decade, he ignored his HIV status (not knowing either way) thanks to the "advice" from contrarian experts on the Internet. One day he finally had a flu that didn't seem to go away. A doctor took one look at the lesions on his face and told him the bad news.

He survived, and now he wants to warn everyone about the dangers of hopeful biases when combined with the immense amount of misinformation on the Internet. He's even appeared on national media here in Finland to tell his story, and I admire his courage.

These examples of ignorance remind me of my own similar anecdote:

I attended a dinner, out of politeness, with my grandparents and their friends. A man staying with them saw me puffing on my personal vaporizer, cornered me, and proceeded to tell me that he was diagnosed with lung cancer after trying to switch to a PV himself.

"I smoked cigarettes for 30 years, never had a problem. Those electric things cause cancer!"

Now, I know there is no long term research into electronic cigarettes or their health effects. However, he was clear that he had only quit cigarettes for maybe 2 months before being diagnosed.

To suggest, with a straight face, that 30 years of cigarette smoke couldn't possibly be responsible for his lung cancer...I'm not sure how he came to such a conclusion other than misinformation.

Edit: hmm, unsure of that grammar. (Non native)

It ran from December 1992 until February 2001, ceasing publication because all the contributors had died of AIDS-defining clinical conditions.

http://www.abovetopsecret.com/forum/thread73547/pg1

yikes.

Take this article:

http://edition.cnn.com/2017/04/20/us/louisiana-climate-chang...

These fishermen seem just like my acquaintance who didn't want to know that his immune system was collapsing, until finally the signs were evident as bright red cancerous lesions on his face.

It's completely infuriating. I mean if I didn't know anything about HIV or medical research, that would sound completely plausible. I went in expecting to read batshit insane conspiracy theories. And I'm surprised how sane it sounds. That's scary.

Academic orthodoxy is sometimes wrong (re: history); But even it is isn't, we all ignore academic orthodoxy in some field of life: diet, travel, wars, entertainment choices, etc...

So who decides which academic orthodoxy we need to force people to conform too? If you leave room for free-thought, you have to expect bad thoughts.

On average the results in science papers are probably more accurate than random stuff you read on the internet. But I suspect the difference in error rate is uncomfortably small, much smaller than those who are smug about science would like to believe.

If you google around about the replication crisis you will discover that the opposite is true.

The sort of knowledge I'm talking about is the kind of stuff that's been known for decades, acknowledged by everyone who's studied it.

Maybe this is a technological problem with a technological solution?

This strikes me as being a sort of blindness to the epistemic web-of-supporting-knowledge. What we claim to know is ultimately supported and reinforced by other propositions that are consistent with it. There is a view of reality defined by the proposition that vaccines do more harm than good, and there is a small web of propositions that stand in support of this; but its a very small web and it stands in opposition to far more propositions than are in its web.

What if there was some kind of page-rank for ideas we see on the internet? What if it was made obvious, in your web browser or something, that you were viewing information that had a very low epistemic probability based on the amount of supporting information on the web? Maybe we're 10 or 20 years out from whatever technological solution could accomplish this (AI, clever algorithms, ... ?) but I really think that sometimes people just have a hard time seeing or understanding the sheer scale of evidence that is opposed to their flawed beliefs.

No, they would discount the credibility of that evidence before entertaining the possibility that they might be wrong.

If you created a page rank for ideas, to prove to them how unlikely their beliefs were to be true, such people would dismiss the attempt as propaganda. Just look at how often various points of view on HN (which one would expect to self-select for rational, scientifically minded thinkers) are derided as fabrications, "fake news", the work of shills, disinformation, or something similar.

There are a few examples in: https://youtu.be/hgEPHIaScck

Doesn't Wikipedia say that climate change is a thing? I bring that up, because it contradicts much consensus.

This kind of things makes me feel sick.

I knew a family, their child got some minor infection, they didn't believe in antibiotics and tried to cure him with "alternative" methods. Suffices to say - that kid is no longer alive.

Edit: I wish they were convicted for manslaughter.

Nevertheless, I take issue with your edit. I believe your wish is in fact rooted in your own desire for agency. Those parents have lost a child, albeit from their own ignorance. In light of this, how do you think their prosecution would be a gain to society? We can surely educate others without inflicting even more damage on those poor, grieving parents.

If they do, doesn't it follow that parents denying appropriate medical care to their children (on whatever grounds) is tortiously negligent and therefore ought to be prosecuted?

"We can surely educate others without inflicting even more damage on those poor, grieving parents."

On the contrary, I think we should make an example of people like this. The penalty for doing something like this really ought to be high enough to dis-incentivize people from carrying out these kinds of actions based on their idiotic beliefs.

I am not arguing that these parents were not idiotic, just that their further punishment can serve no purpose.

We may have to agree to disagree though.

As an individual, you have most probably a huge number of things you just believe and never try to challenge. And among that, many that, if challenged by someone else, you would still hold to.

All in all, you can't start to change the world unless you recognize your own failures in the other people. They may do stuff you know are perfectly dangerous. And you may know better for a lot of things. But in the end, you are like them, and need to see it instead to have a real influence instead of thinking you are above that.

Absolutely, there must be things I think are true which are not. And particularly there must be opinions I would find hard to change. I don't deny that.

For instance the whole religion thing, I'd probably take it with a rather large grain of salt if god appeared and started giving evidence for himself, even though I think of myself as someone who is swayed mainly by evidence.

But I also think, perhaps wrongly, that we can actually learn things about our world, and the way to do that is roughly what we call science. And currently, a lot of people simply cannot think in this way. They don't have the evidence that hordes of scientists have gathered, and they don't possess the thinking skills to process what they're told when they come across new evidence. Irrespective of what my own personal knowledge gaps are.

>All in all, you can't start to change the world unless you recognize your own failures in the other people.

I don't believe in this pseudo-psychological idea we need to understand ourselves before we can learn anything about the world, or start to change it. Plenty of people have changed the world for the better without themselves having a complete understanding of their own biases. How could we know anything, if not in small pieces?

I disagree. I think that people contort themselves to find ways to see themselves as internally consistent precisely because they are rational within the inescapable prison of their own perception.

The issue really comes down to causation/credibility. Human psychology will cause each person to strongly favor whatever causal explanations best satisfy their biases.

Your example with religion is a good one. For purposes of this argument, let's assume that event actually did occur and it was not a hallucination.

To a religious person, the event would be a well-deserved manifestation given as a reward for their years of faithful observance, and they would likely continue stronger than ever in their convictions, fully satisfied that their position had been irrefutably proven.

But to a strongly non-religious person, the event you've described would be interpreted as a mental breakdown and cause them to seek medical treatment. After being duly medicated, this person would also likely continue stronger than ever in their convictions, having personally "overcome" a hallucinatory episode.

The same objective event occurred, but the context of one's psychology makes all the difference in its perception. This works with practically everything. No data, no event, no observation exists in a vacuum.

In the case of science/academics, people with agendas and biases are involved every step of the way, from designing the inputs to summarizing and drawing conclusions from the results. And there have been, and surely will be in the future, many significant incidents where the accepted consensus is proven disastrously wrong. Before that happens, the small handful of contrarians attempting to warn about or discuss them are typically considered fringe alarmists, bitter saboteurs, or establishment puppets.

This creates logical space for anyone to disregard the naysayers, set up camp in the contrary position, and gear up for an "I told you so" moment down the road.

If you know this person to be wrong, you will not win by attacking their conclusions, no matter how right you are. Rather, you will win by properly estimating their perspective, understanding why they choose to discredit the particular authority that proves out your position and credit a separate one that doesn't, and then enlightening the situation for them such that their own natural reasoning process will obtain to the "correct" conclusion.

This is called "sales" when it's done on a 1:1 basis and "marketing" when it's done en masse. It can be very difficult when there are powerful entities competing to cast the psychological context in their favor.

Uh, marketing hasn't attempted to provide any arguments that attack conclusions, for a long time. It's been primarily about creating subconscious associations that are completely separate from arguments.

I find this a really scary thought. It's not that people can't because of their age, but intuitively we know that people just won't be bothered to put in the effort. Especially when they've spent their whole life in a certain frame of mind. What's the incentive to change after so much time? For the vast majority of people there will be none, and that's scary.

Don't forget to mention that one of the early proponents was involuntarily committed and killed as a result of social exclusion by his colleagues.

Later history was rewritten and a diagnosis of syphilis was used as justification.

https://en.wikipedia.org/wiki/Ignaz_Semmelweis#Breakdown_and...

The internet has changed things in that it allows one to obtain information that appeals to their biases in a totally isolated and exclusionary manner.

Like it or not, if you're sharing space with someone in meatspace, their actions, choices, and reactions are going to be observed and processed by everyone else in the same space, having some type of influence, and the only way around that is for one of the parties to leave the physical area, which is usually not a trivial thing to make happen.

On the internet, if someone exerts an influence that causes discomfort for you, you just press "Mute"/"Hide"/"Block"/"Close" and it's gone forever. Our platforms learn that your biases oppose that type of influence and of course, tailor your experience to better appeal so you don't choose to use a competitor's platform.

The same kind of thing happens with search; someone who wants to reinforce their anti-vax bias will search for "reasons vaccines are bad" and find just what they've asked for. Someone pro-vax will do the same. The truth is that very few people in either camp are legitimately qualified to seriously critique either side's position, since you'd have to be a drug chemist yourself to know if the claims are valid, so as long as it looks like some semi-credible mumbo-jumbo at the surface level, it's good to go and the user will run with the one that works in favor of their bias. They never have to go through the discomfort of hearing from the other side. It's much harder to get this type of isolation in the real world for several obvious reasons.

The net effect is increasingly isolated social bubbles, which means increasingly diminished tolerance, which means increased hostility and polarization. Others can speculate on what comes after that, but it's probably not good.

Sure, that's possible and common. But it's still just as easy to mislead secular persons as it is to mislead religious people.

>one can make a moderately successful effort to reduce these weaknesses somewhat through disciplined practice in critical thinking

Yes, I agree and I think that's important, but it should be acknowledged that wherever you're dedicating energy to this effort, it's coming out of something else, and that when you're not aware, you are basically falling back to default decision-making and credibility determination mechanisms. One should not make the mistake of believing that because they are good analysts, their decisions automatically qualify as more deliberate.

>Additionally, the scientific method itself is engineered to reduce the effect of these natural human biases that will always plague the scientists themselves. Religions, on the other hand, consistently enhance and exploit these biases to promote their narrative.

This is where you're starting to lose me.

Academia is just as vulnerable to bias and groupthink as anything else. It is laughably naive to say that the principles of the scientific method keeps academia more objective than any other field.

In fact, the opacity and density of academic papers will frequently be used to obscure what is otherwise plainly false information, allowing it to be weaponized into propaganda (with varying degrees of subtlety) that common sense and/or decency would not allow people to accept any other way. "There are lies, damned lies, and statistics", as the saying goes.

"The diocese says..." and "the university says..." are identical lines of thought, applied to a different authority. In both cases, the technical background behind the proclamations from either institution is serious and well outside the grasp of the average devotee or proselyte.

You may be operating under the belief that religious proclamations occur based on the whims of religious leaders, but this is no more true than the claim that academic proclamations occur based on the whims of academic leaders. Both groups have significant influence and control, but people do not just follow these things without an apparent technical/rational justification, and credible religious edicts are works of real and serious scholarship.

I don’t equate “academia” with the scientific method. In fact, the two are often opposed to each other. But since that is where you want to go with it, I would question this sentence of yours as a false equivalency. I submit that “academia”, as much as you can think of it as a single institution (of dirty liberal elites or whatever), is definitely vulnerable to bias and groupthink (even to an astonishing amount, depending on your preconceptions), but your idea that it is just as vulnerable to bias and groupthink as literally anything else is not true. For example, I can substitute “flat-earthers” for your “anything else” and it becomes very clear that flat-earthers as a group are much more susceptible to bad ideas than “academia” as a group.

Merely pointing out the failings of scientific practitioners, of which there are copious amounts, is not an argument for putting the scientific method on equal ground with religion. Appeals to authority is not equivalent to the scientific method, and it is not the only reason why one would trust it over the declarations of religions. Perhaps you might be one of those people who thinks that because science refines its scientific theories over time, that they are “always changing their minds”, rather than creating more accurate models. Maybe you might be one to say that evolution, for example, is “just a theory”?

One can easily see the kind of bullshit that exists within the scientific community without being so unwise as to think that makes it just as credible as any other source of information. I could talk your ears off about what annoys me about the research community, but you’re probably fairly well versed in that, since you make it an important part of discrediting them as a whole. Ultimately what annoys me about them is precisely the ways in which they deviate from the scientific method in order to advance their goals of getting published and recognized. There are bad incentive systems in place in academia that make it so that papers with no real substance get published; negative results are polished like turds to pretend like they were looking for a different result the whole time. Bad statistical methods are used to obfuscate; no actual reproduction of results are being done in large swaths of the field; scientific journalism hypes single paper results to the detriment of science in the public’s eye. I could keep going. Yet still these failings do not even hold a candle to the actively misleading aspects of religion.

You may be operating under the belief that religious proclamations occur based on the whims of religious leaders, but this is no more true than the claim that academic proclamations occur based on the whims of academic leaders.

Many religious proclamations actually are made up whole cloth by religious leaders. Take Joseph Smith, for instance. When someone invents their own religion, it is called a cult in the beginning, but that pejorative eventually goes away after the founder dies and the organization evolves and achieves more mainstream success.

However, it isn’t terribly important whether religious proclamations come freshly made up from a single person or if they are evolved from legends and handed down from earlier generations. What’s important is their epistemological foundation. “God says so” seems to be the sum of it. The evidence? Emotional reasoning and appeals to just having faith.

credible religious edicts are works of real and serious scholarship.

I don’t know what “scholarship” is supposed to mean in this sentence. Just because a person studies something does not mean that the studying was done properly or that the subject matter or learned material has any merit whatsoever. I’m not aware of any religious edicts that have revealed any new knowledge to the world and have met their burden of proof. I am aware of a whole lot of edicts that have not.

It's funny you mention Joseph Smith because he disproves your case. Whether someone believes his claims or not, he offered something to substantiate them: a lengthy, coherent book with a reasonably-consistent internal narrative that claims to underpin his theological innovations.

Same is true of L. Ron Hubbard; Scientology is a thing that has gathered a following and continued on solely because he provided something substantial to anchor it against.

Thousands of rambling fringe religionists have come and gone since Joseph Smith and they fizzled out precisely because they had nothing significant to undergird their claims. The small followings these people gather out are held together by force of personality, and as soon as they're gone or diminished in the group's eyes, the group dissolves.

>However, it isn’t terribly important whether religious proclamations come freshly made up from a single person or if they are evolved from legends and handed down from earlier generations. What’s important is their epistemological foundation. “God says so” seems to be the sum of it. The evidence? Emotional reasoning and appeals to just having faith.

To the layman, the evidence for either is the equally convincing. Compare Clarke's Third Law: "Any sufficiently advanced technology is indistinguishable from magic."

Joe Q. Public truly has no idea if the claims of a scientific paper have a valid epistemological foundation. One must undergo many years of training to become a subject matter expert and provide a serious or credible critique/endorsement of such works.

Until that point, the reader is at the mercy of his credibility heuristics to determine which subject-matter experts appear to know what they're talking about.

Serious religious works function the same way. The average member of the public cannot go into something like the Book of Mormon and make an extensive critique of it, weigh its archaeological consistency against similar theological documents, etc., just like they can't go through a scientific work and validate that the scientist sampled his/her data wisely, chose the appropriate calculations, accurately summarized the data, and so on.

If you have the dense tome for people to gloss over and use to make their own connections, it becomes 100% about projecting an image of trustworthiness and believability.

You can say "Well they can see my science works", but then, the religionist could also say "well they can see my religion works", and then it's all about impression/perception. For example, one could perform a magic trick and claim to have invented a hat-based rabbit replicator. Provided you had enough bunnies and interactivity to fool the senses, this would be highly persuasive unless there happened to be someone who was competent to get into the mechanics of it. "Seeing", or thinking you're seeing, is believing.

People are, in fact, much more excited to experiment and engage with religious content because it addresses an area of their lives that they feel is much more important and accessible (their personal/emotional/familial well-being).

Religion and science are much nearer to each other than you think. Their separation is fairly recent; before the last few centuries, it was typically religious institutions driving innovation, understanding, and discovery.

Religion and science both must be accepted based on trust heuristics (called "faith" in religious contexts) by everyone who is not an expert in the specific speciality, which means that everyone is relying on these 99% of the time, since one can only be expert in a small portion of subjects.

Because acceptance of these principles operate identically, you can hijack either group by using the same method: produce a tome that is surface-plausible, and then change things to match the trust heuristics most commonly deployed throughout the group. This will require some changes in framing and vocabulary, but not in the fundamentals.

I think you're saying that since humans have limited capacity for time, attention, and learning, and since there is so much information and claims out there in the world, we all naturally have to rely on what you call trust or credibility heuristics in order to determine what we will accept as truth and what we will reject. They are heuristics in the sense that they are shortcuts that make up for the fact that we cannot go and learn everything there is to know about every subject that crosses our path, and these shortcuts are necessary because we must make decisions every day concerning what we will believe when faced with claims and new information regarding subjects we know precious little about. These shortcuts or heuristics often take the form of placing trust in previously accepted authorities on the subject, the opinions of trusted acquaintances, or one's own senses.

I agree with all of that.

Further, I think you're saying that the credibility heuristics that people use are all the same, and are equally effective in determining truth from fiction, and equally vulnerable to being hijacked by con artists with books and large followings. This therefore must mean that people who accept scientific consensus on topics like evolution do so for the same reasons that people think Joseph Smith talked to angels.

I would dispute that.

This goes back to my previous comment where I said that it is possible to find oneself on more epistemologically solid ground than another on a given topic. This is possible because some credibility heuristics are measurably better than others, and one can improve one's own credibility heuristics with the application of standard critical thinking skills.

So maybe you have an authority you trust on a given topic X. One question to ask yourself is, why do you trust this person on topic X? There are good answers to this question and there are bad answers. A good answer might be because this person has a degree in X and has worked in the X field for several years. A bad answer might be because this person knows a lot about topic Y, or perhaps because this person talks about X a lot and claims to know about X. A well trained set of credibility heuristics seems to avoid potential problems here. One type of heuristic is more likely to be hijacked by bad information than the other.

Perhaps you accept proposition P. You might ask yourself why you accept this proposition. A good reason for accepting proposition P would be that proposition Z is an observable fact, and that Z => P is a result claimed by several diverse trusted sources. A bad reason would be that P => Z is claimed by several diverse trusted sources, and that Z is an observable fact.

A proper set of credibility heuristics would also take into account the weight of the claim, namely a measure of how important and/or extraordinary a claim is. For example, if you were claim in your next comment that you own a cat, I would accept that claim at face value. The reason I would so easily accept this claim is that it is both mundane and unimportant. There is no risk to me for accepting this claim if it happened to be false, and it is an entirely ordinary thing to own a cat.

Now if you were to tell me in your next comment that you were abducted by aliens, then I would dismiss the claim. It is an extraordinary claim which therefore requires extraordinary evidence that cannot be provided in this medium, and accepting such a claim would have a huge effect on my worldview, and therefore incurs considerable risk on my part. Any credibility heuristic that fails to account for this variance is going to be more susceptible to being hijacked.

With basic application of critical thinking, one would understand that a hat-based rabbit replicator is an extraordinary groundbreaking claim requiring extraordinary evidence, including a full explanation of the mechanism, independent verification, and a consensus from a variety of diverse external sources. One would understand that the established truth of such a claim would revolutionize science almost overnight and would have a huge effect on the way we all live. It would be a extremely difficult task to convince a critically thinking person using merely "enough bunnies and interactivity to fool the senses". We understand that the senses can be fooled in various ways. We go to magic shows specifically for this reason. A good credibility heuristic would take this into account.

There are a myriad of critical thinking tools that can be used to improve your credibility heuristics in a way that does not require you to become a subject matter expert in every area that affects you. To the extent that you pursue these techniques and apply them, the less likely you are to fooled by false claims dressed up to look plausible. You can avoid falling for conspiracy theories like the flat-earthers, the moon landing truthers, and Holocaust deniers.

Well tuned credibility heuristics would help you understand that a story appearing to be "coherent" or "reasonably-consistent" does not support the claim of it being a translation of ancient golden plates scried from a magical peepstone. It would also help you avoid falling for the idea that praying about it and having a good feeling afterward is evidence that a ghost is telling you that all of the claims are true, or that a lot of people believing in a set of claims is by itself evidence for the truth of the claims.

Everyone's credibility heuristics could be improved by understanding that Sturgeon's Law still applies to individual scientific papers hyped by the media, and that independent repeated verification is a necessary filter that takes time and patience. A basic understanding of how the scientific method works, why it works, and how it is generally applied in academia and research institutions (warts and all) is an important part of a healthy set of credibility heuristics.

Carl Sagan called these types of well-tuned credibility heuristics a "baloney detection kit". He has a whole book about it, which I highly recommend.

Well, they'd pass through your village at regular intervals. But I understand what you're saying - in the old days, a snake-oil salesman had to travel through each village, and risk his business and life. Today, you can hire a botnet to reach ten million people with your claims of a magic stock about to rise, or a pill that makes you irresistible to women, or a device that cures your cancer.

It must be understood that this is the case with all mentally competent people. An ego's appetite can be trained and controlled, but generally speaking, everyone wants to see themselves in the same basic, overwhelmingly positive way.

When an "empiricist" comes into the fray, they've trained their ego's appetite to prefer circumstances that allow themselves to be on the side of "hard data", "scientific consensus", or, as you've put it, "academic orthodoxy". Thus, the empiricist will dismiss theories that seem to be "outside of the orthodoxy" and ridicule their purveyors/supporters/believers.

This is your ego operating to give you the room to feel superior and keep your self-image adequately positive. The same thing is happening in your friend who has the quantum healer machine. They see themselves as someone who is able to spot frauds even when many other people cannot and use this information to their advantage (something you have in common).

Most likely the key difference here is the amount of trust each party has learned to place in secular/technological/academic institutions. You believe you can see the fraud perpetrated on the public by exploitative marketers and snake oil salesmen. They believe they can see the fraud perpetrated on the public by exploitative elitists and the corporate upper class (that's currently suing them for an unpaid medical bill).

The truth is that both perspectives are sometimes valid. "Learning how to think" is learning how to recognize your biases and think about an issue from a neutral, dispassionate perspective, without discounting any potential argument without consideration.

It's never too late to learn that, but most people have internalized their emotional disdain for "the bad guy" that they believe is oppressing them, whomever they assume that is, such that they'd rather make up a justification to discredit the other side's arguments rather than admit that they're potentially, though not necessarily, reasonable. And while overriding this impulse is a great skill to practice, it must be understood that very few people will ever do so to an extent that makes it usable.

---

The key to understanding people, IMO, is recognizing that human decision-making does not operate on objectivity or observation, but rather on the unavoidable and undeviating need to see oneself as intelligent, aware, and discerning.

People must see themselves as important and significant to justify their existence. This is true of all people; it's part of human self-preservation. Do not believe there are exceptions. Most people don't like this explanation because it implies that their self-importance is illusory, which funnily enough, really only reinforces its correctness.

The way to control your ego is to train it to value the right things as signals of importance.

If you want to change someone's mind, you must not argue that their conclusion is incorrect; rather, you must discover what biases and emotional mechanisms are at play in shaping the perception that leads them to that conclusion.

Once you accept this, the world is much easier to understand, social situations are much simpler, sales and marketing become both much more practicable and much easier to notice, and a lot of bitterness and antipathy goes away.

Disclaimer: I have no research or supporting evidence, this is pure conjecture, and there is no reason at all why you should listen to me.

Damn... while I was reading your post I was yelling "I buy it! I buy it!" in my head, and then it turns out you don't have anything more to sell? :(

There are many other examples of pre-internet mass gullibility, like homeopathy, Reiki and assorted foolishness. Maybe the internet has made it easier to market foolishness, but I doubt we can say it has made us more foolish or gullible.

No, but it does help connect everyone with the particular flavor bullshit they're going to enjoy eating up.

However it also makes it easier for them to organize and spread their ideas...

I'd love to see data on wether this is a net gain/negative.

you could think the moon is square, doesn't bother me one bit. i'm living my life, not yours.

it's far better to focus on your own knowledge and self improvement. it yields a higher return on investment.

if you want to affect change, throw your time and money into actually influential things, not social media arguments.

I think a lot of people need to be educated about what the benefits of debate are and how they manifest slowly over time.

Care to give any examples and the reasons for your first position and subsequent change?

I also liked to think analytically and enjoyed the intellectual stimulation I got from certain online forums (this one included). I liked how people would trade counterarguments and keep each other honest by pointing out bad thinking or logical fallacies. When someone would argue against something I believed in, I would argue back. From their perspective, my mind was not being changed and they were wasting their time, but from my perspective, the things they would call me out on stuck with me. I tried to avoid falling into logical fallacies and bad arguments because I knew I'd be called out on them. This restricted the ways I had to justify my beliefs, so that made it challenging. Then at church I noticed that nobody was being held to these standards at all, and fallacious reasoning was being let fly freely and constantly. I experienced a lot of cognitive dissonance during that point in my life which ultimately led to deeper investigation and study. I give a lot of credit to honest online debaters who pushed back on my ideas and kept me honest about the substance of my arguments.

The problem is when this wrong headedness starts to interfere at a public policy level, like climate change.

What I mean is, it doesn't even have to affect public policy to be bothersome. It also affects informal small-talk with acquaintances.

When people have clearly wrong beliefs (e.g. a square moon), it might not matter all that much to you, but they might harm someone else by spreading the trash, or worse, connect with like-minded people and create a real danger for others (legislate pi to be 3). That mind of trash is no less a part of your environment than the physical street, and I think it's a very good thing if you make an effort to pick up the trash, even if it's not yours.

And you learned the wrong thing. It is possible to change peoples minds and behavior with online conversation. It just isn't easy and it doesn't go quick (a single line directly pointing out a flaw often isn't enough, you have to bring across that you exerted effort, to show you care). If over time enough people invest time and effort into pointing out that you are wrong in something, you will change your mind. You might not admit it, you might not even notice it, but it will happen.

You might be right that it is perhaps not the fastest or most efficient approach, when you suggested spending time and money on actually influential things (though I'm not sure what you have in mind there).

Just look at the responses you have received so far here. Several people with different backgrounds have responded to you and said "you are wrong, you can change people's minds in online social interactions".

Now you have a choice. Will you hold on to your (really not very good for mankind as a whole) statement, so that it self-fulfills and you can still consider your own opinion correct and your thoughts and behavior consistent?

Or will you take that leap and believe people can change and act accordingly going forward?

Edit: I hope you realize that if you respond to my plea with a convincing statement and I believe you and change my mind, we both enter seriously troubled waters.

After all, do you really want to change my mind on the fact that one cannot change someone's mind in a social media setting, in a social media setting?