I don't want a search engine removing whatever they deem a "conspiracy theory" or "misinformation". I can decide on my own.

> I can decide on my own

What tests have you done to confirm that you're actually any good at this?

Or is the idea that you'd actively prefer to have your life ruined by misinformation than admit you might not have the best ability to distinguish - like the way some people would prefer to be crippled or maimed than accept safety protection on dangerous machinery because they find it emasculating?

>What tests have you done to confirm that you're actually any good at this?

That's where the "decide on my own" part comes from. I don't need to pass your tests or arbitrary criteria.

The most important point in having a democracy and citizens worthy of the name, is that you don't delegate this sorting out the wheat from the chaff to some third party that "knows better".

What makes you think a multinational corporation would be any better? They are at the mercy of regulators and will always bow to political pressure rather than the truth.

> What tests have you done to confirm that you're actually any good at this?

I can figure it out. I don't need to get information filtered by people who think they know what is best for everyone (else). This is what China does.

Most likely, if you're posting on HN, you're probably at least OK at critical thinking

Isn't that essentially saying you don't want your search engine ranking by relevance and quality?

Quality, the same "AI" recommends me irrelevant videos, cant recognise me as a human if i log in from any browser other than chrome, and that issues takedowns randomly for users. The ine that spams, pinterest like shit site quality on front page. Yeah, no, I dont want it to decide.

The first thing I do when searching for images is `-site:pinterest.com` The quality of results increases immensely.

IIRC Google used to have a dropdown in the search results similar to "Don't show results from this site". Luckily I don't see Yahoo Answers results anymore but would be nice to have Pinterest removed.

If you use Greasemonkey / Tampermonkey, etc. you might want to check out 'Google hit Hider by domain'[0] [in spite of the name, it works on several search engines, not just Google]. It provides a really simple interface to ban [with various levels of extreme prejudice] sites from ever again appearing in your search results.

[0] https://greasyfork.org/en/scripts/1682-google-hit-hider-by-d...

This looks great, thank you!

No, it's like saying I want the search engine ranking by relevance and quality of search terms and matches.

If I search "we never landed on the moon" I want to see results about those theories, not see them censored because the search engine "knows better".

These days big tech seems to be focused on hiding certain information from users, herding them into a certain echo-chamber.

No thanks, I'd rather think for myself.

Relevance and quality are subjective.

Of course they are. That's the reason Google "won" in the first place after all. They had a algorithm which better fit what people expected.

But tides are turning, Google's results are steadily going downhill. Unfortunately it seems to be too much of a daunting task with today's size of the internet to jump in with a viable competitor.

> But tides are turning, Google's results are steadily going downhill. Unfortunately it seems to be too much of a daunting task with today's size of the internet to jump in with a viable competitor.

I wonder what portion of the internet you have to scrape to be able to have good results for 90% of queries. An enormous number of my queries end up on Wikipedia somewhere, for example. A lot end up on Stack Overflow. The two of those probably constitute the majority of my searches, and almost certainly a majority of the searches where I really care about the answer (and I'm not just googling whether pirates and ninjas were contemporaries of each other, because why not).

If Google's results are really slipping that much, you can make up for the lack of data by providing better results on the data you do have.

I am curious why something like that hasn't gotten more popular. A search engine that only indexes highly reputable and public sources would be interesting. Wikipedia, dictionaries from somewhere, reputable newspapers globally, etc. Is it just too big of a target for abuse to be viable?

Good points.

> I wonder what portion of the internet you have to scrape to be able to have good results for 90% of queries.

But this is it. I don't think it'd be too hard to get 90% of queries right. It's the 10% that are the challenge.

"Page rank" / "link juice" (amount/strength of incoming links) is still a main ingredient in Google's recipe, so they are actually able to determine, without manual intervention (more or less), which sites are "reputable", while your suggestion would require someone to curate the assortments of "reputable" sites.

How would you discover niche blogs? Or even more high profile ones, just not known to the major public?

That said, as you say, you could probably do with something like this for most of your day-to-day searches, and then just !g or whatever for the rest.

What criteria should be used to rank search results?

Search engine should give access and filtering to all metadata so you can narrow your search as you want.

Ok, but a search engine has to sort by SOMETHING

Right, but sorting results isn't the same as removing a subset entirely.

Putting something last on a search result of thousands is basically the same as removing it entirely. Hell, even putting it on the second page is basically equivalent to not returning it at all.

>One thing I found is how bad ddg is at removing conspiracy theories and other misinformation.

In all kinds of regimes, from dictatorships to western democracies, all kinds of valid (and sometimes proven in court, or historically, later) stories have been officially called "conspiracy theories" and/or misinformation.

And inversely, in all kinds of regimes, from dictatorships to western democracies, all kinds of bogus state propaganda has been officially called "the truth".

I'd rather not have a search engine decide for me...

Speaking as somebody who goes with mainstream expert opinion by default, I nevertheless think there's enough censorship from the big tech providers, at the very least let's have such "filter out stuff that annoys us" initiatives as optional.

I can’t believe anyone is in favor of censorship in the first place. Maybe they just need to experience the harsh reality of rule by majority opinion or mob rule. Reddit is the perfect example. I can’t tell you how many times a top up voted thread or comment is 100% false. The experts aren’t the ones voting is the problem so who gets to the top is only the one most persuasive or with a comment that already agrees with expectations. This is exactly how things work with politics or news as well. I can’t tell you how many news articles I’ve read that make extremely basic mistakes and assumptions because the person writing the article has no experience in that field.

But there is space between outright censorship and deliberately building a system that promotes the ideas of those who are the best at information warfare.

If you repeat your idea often and loud enough, you can make a large number of people believe anything.

Internet curated by algorithms is a perfect tool for spreading misinformation:

- You need not to carry the weight of any of the legal structures or responsibilities of a journalist.

- You can produce misinformation faster than the news cycle is able to react, meaning your misinformation will be able to poison the well before any conversation is able to take place.

- You can segment your audience so that dissenting opinion is nowhere to be seen.

- By the time any debunking by actual journalists can be done, your target audience is already exposed to the next cycle and can barely remember anything about the previous one.

This is extremely dangerous.

I am sure that the people who came up with automobiles weren't trying to create a vehicle of death, but there needs to be some kind of safety check akin to seat belts for the information propagation that takes place online. I am not sure what the answer is, but acknowledging the problem and trying to do something shouldn't be dismissed with a mere cry of "censorship" when information can and is being weaponized in a way and at a scale that has been impossible so far.

And who is fit to play the censor? And who shall determine if a group is full of bad conspiracy theories or ok ones? And how shall we address it when the censor is wrong?

It was not long ago that trans and gay people were censored "for the common good".

Let's not repeat those unfortunate lessons.

Those are excellent questions. But if your freedom of expression collapses itself, it won't survive anyway.

The Nazi party got into power in Germany democratically. They used the privileges given to them to take those privileges from others. Give people the freedom to elect their representatives with no checks and balances, and the results can be ugly.

The polarization and brainwashing happening through social media today makes me think it is completely feasible, if not inevitable, that a group will form that will enact actual censorship to the internet.

In the settings on search they currently only have the option "hide explicit results"; that could be expanded to give users control over the filters used.

Google has to choose something to prioritize. It's good that in this case it chose the authoritative site over the conspiracy theory one. Even better if they chose it because it was more authoritative rather than because they calculated they could show you more ads that way, but I won't give them credit for that just yet.

They didn't censor the other site completely. If I search with site:dangersofvaccines.com I get results from it. I can't get results from it any other way though, including searching text that only appears on that site - they'd rather tell me there are no results - so I agree that's an unpleasant level of censorship/nannying.

Some people are fine with making decisions for themselves and don't need a net nanny to say "mummy thinks you shouldn't be reading this".

I agree in general. But what if I did want to know if vaccines are dangerous, so I search "are vaccines dangerous". If I want to find an anti-vaccine site, I should be able to do so, but the search term "are vaccines dangerous" should be viewed neutrally, in which case I would find links from cdc or .gov sites to be more useful that anti-vaccines sites

That's fair; I suppose it's inevitable that search engines move more in a similar direction to what we know as the main high street when it comes to high profile searches, keeping more mainstream stuff while losing the off the wall areas but it does seem a shame to loose some of the weirder side streets in the process (the sort of weirdness from the early 2000s internet for example).

It's a bad idea. There are examples where mainstream was wrong. Usually when it's about big money. E.g. decades of anti-cannabis propaganda.

Now you mention it, the official position on the correct response to coronavirus flip flopping and changing from location to location is a good recent example too.

We go to school in the US for usually at least 12 years. Any person should already be educated on topics such as anti vaccination. In particular in my state you can’t even go to school if you haven’t had your vaccines. Not only that but we learn how to learn in school. The argument for censorship against stupidity assumes exactly that, that the majority of people are stupid which they are not.

> The argument for censorship against stupidity assumes exactly that, that the majority of people are stupid which they are not.

It's not censorship against stupidity. It's censorship against bullshitters who spread medical misinformation that may put people in danger; factually untenable opinions, expressed in an assertive manner, in a context which seems authoritative to the layperson.

Nobody says the majority is stupid, nor it is relevant. Protecting a minority is as good an excuse as it is protecting the majority, don't you think?

My point was that we should and can understand the difference between sources of bad information and good information, this is and should be taught in schools. I don’t think we should protect the minority by limiting everyone. We could make a safety vs security argument but in my opinion we should protect children only and allow adults to make stupid mistakes. This is why we have drinking age, smoking age and drivers license ages. At some point we need to assume a level of intelligence in the population and in accordance allow freedom to make mistakes.

> Any person should already be educated on topics such as anti vaccination.

The quality of US public schools can vary immensely, especially for students in poor areas where the community can't put up much money and education budgets are the first thing to be slashed.

Additionally, many states allow outright lies such as creationism or abstinence-only sex ed to be taught - both of this is further evidence that it cannot be assumed automatically that a person educated in the US is well educated.

Yes there are stupid people out there but it is not the majority, even in poor areas as you are claiming. Let’s say you are even correct that poor people now are too stupid to understand anti vaccination groups are not science based. The solution is not censorship of anti vaccination groups, it is education campaigns to counter this. Education in schools, commercials, websites and social media. Not censorship of bad information.

That would require that people are actively interested in being educated. And to be honest most people aren't even in school, much less when they're grown up.

To make it worse, antivaxxers explicitly mistrust science and only trust the blogs and "mama groups" and other crap. How is anything the government, scientists, the media, anyone supposed to overcome that mistrust?

Vast parts of the population however are not, as evidenced by the results of the last election or the support rates of stuff like qanon.

Please keep in mind that qanon, antivaxx etc. are active hostile infowars campaigns. It is a duty of a society to protect its citizens against such efforts.

With all due respect your comment history outlines you as a progressive and the first line you have on your profile is "You call it "alt-left", I proudly call it "Antifa".".

Among the many aspects of your faith that I disagree with, one of the most utilised by its followers is its enthusiastic advocacy of false accusations against heretics in the pursuit of attacking them. And so while I don't know anything about qanon or infowars, your bad mouthing them is only lending them credibility; it is also shores up my belief that whatever your implicit definition of "duty of a society to protect its citizens against such efforts" is, I can assure you, we neither need nor want it.

> it is also shores up my belief that whatever your implicit definition of "duty of a society to protect its citizens against such efforts" is, I can assure you, we neither need nor want it.

I'm not just an antifascist, I'm German. My ancestors proved for the rest of the world where a worldview based on lies and hate (and qanon is nothing more than that) will lead. Their history is proof that a society must defend itself or it will collapse.

Again, your faith believes false accusations against perceived heretics are acceptable tactics; your word against them is less than empty.

Your ancestors and plenty of other examples throughout history are proof where censorship and book burning lead. The principle of sunlight is the best disinfectant has proven time and again to be true and the only people who argue against it are those being shown to be wrong under the light. Yes it permits the odd crazies to exist on the fringe, that's fine; flat earthers anti-vaxers and the rest were pointed at and laughed at long before the recent desperate grab for censorship by authoritarians was made.

Open access to information is only a problem in the eyes of those who want to misslead.

> The principle of sunlight is the best disinfectant has proven time and again to be true and the only people who argue against it are those being shown to be wrong under the light

I am a strong advocate for free speech, but I really think it is naive to believe that truth will always win no matter what. History is full of examples where propaganda and lies wins. I am not arguing for censorship, but I am arguing that we need to confront lies and propaganda with more than just platitudes about the truth winning out in the end.

The truth needs advocates.

That's a fair point; I know there's a lot of truth in the quote “A lie can travel halfway around the world before the truth can get its boots on.” and I've seen plenty examples of major damage caused thanks to the phenomenon (Which has no doubt helped with the arguments advocacating for censorship.)

I'd say a couple of things.

In a situation where we do have free speech I believe the truth does overall win out in the end; very often it takes a frustratingly long time and often comes too late to repair the damages caused by the lie but I believe that that overall, if you'll excuse the horrendous remark, "the truth does find a way".

I also know that the current lies and propaganda we so often see are an absolute stain on these past decades and I'm also in agreement that something needs to be done. Whatever it is however I'd like to see it built on the free speech principle rather than against it, even if the principal isn't without its flaws.

One thing I would say is that while it might be true that truth eventually finds a away, the lies can do a lot of damage in the meantime, and can in fact be used to prevent the truth from coming out.

For example, take fascist propaganda... the lies are effective, and if left unchecked, can convince people to support fascist leaders, who will then block free speech, preventing the truth from winning out.

This is a form of the paradox of tolerance..

Perhaps the fixes for the problems we have shouldn't involve some patchwork bodge to the laws that align our principles but should instead start at investigating why we need these bodges in the first place? It's an unthought out idea but between existing slander laws, a reasonable expectation of common sense/reason from your average citizen and a lack of corruption in government; I'm wondering if setting up the systems to ensure those aforementioned areas are solid would take care of the free speech problems for us.

> a reasonable expectation of common sense/reason from your average citizen

Half the US has voted for someone explicitly denying science, half the UK has voted for Brexit, in some areas of Germany 25% vote for people just a few beards short of Nazis, Bolsonaro is still in office. The "average citizen" can not be trusted to vote in their own interest or in the interest of the future generations.

> The "average citizen" can not be trusted to vote in their own interest or in the interest of the future generations.

Maybe not, but in a democratic system the right to choose trumps any ideas about what you or some elitist group thinks is best for future generations.

Are you advocating for undemocratic system?

For the third time.

> your faith believes false accusations against perceived heretics are acceptable tactics; your word against them is less than empty.

Cool, I’ll see about doing searches with it more often to get a better read on quality differences :)

It’s interesting to me that despite the widespread complaint about the bad quality of DDG (which I, excepting long-tail results, disagree with), at least for my quick checks of "ddg" and "css center text" from your suggestions, I prefer the DDG results in both cases (especially for the css search).

I searched for "are vaccines dangerous" in ddg, and the results are reasonable.

https://duckduckgo.com/?q=are+vaccines+dangerous&t=fpas&ia=w...

The first page of the US search results: vaccines.gov, chop.edu, cdc.gov, livescience.com, scientificamerican.com, nytimes.com, newsweek.com, healthline.com, who.int, mayoclinic.org. That's a good mix of the official outlets and some popular explanations. Results change of you specify a different region or no region at all. For reference, these are results in Italian, many of them are rebuttals of a minister claiming that vaccines are useless and dangerous: https://duckduckgo.com/?q=vaccini+pericolosi&t=fpas&ia=web

But if you try to search English terms in a non-English region, expect ask kind of SEO garbage.

> One thing I found is how bad ddg is at removing conspiracy theories and other misinformation.

You should probably blame Microsoft for that. I mean DDG is just a Bing API wrapper.

Yes, that’s why [0] and [1] show the same results. Only, they don’t. They are more similar than DDG vs. Google, but not the same and DDG has (for me) more relevant results closer to the top.

[0]: https://www.bing.com/search?q=css%20center%20text

[1]: https://duckduckgo.com/?q=css+center+text

Weird. The results are in the exact same order for me when I clicked both of your links. The only difference is that the bootstrapshuffle.com result is below (or rather, on the second page) the stackoverflow result on Bing. Considering the former is specific to Bootstrap, it's less relevant than the stackoverflow result.

Anyways, DDG does mainly use Bing, but they also do their own modifications to the rankings, which is sometimes for better and sometimes for worse.

> [...] but they also do their own modifications to the rankings, which is sometimes for better and sometimes for worse.

Could you maybe share a reference for this?

I'm a couple of days late to this, but no, I cannot provide a reference (anymore). DDG used to have a page that described in pretty great detail how the results were sourced and ranked, but it seems the current page is buried pretty deep and has less information specific to ranking. IIRC, the Bing results (and previously Yahoo/Verizon Media) rankings were augmented by their own crawler (DDBot) and aggregated usage of their search. I believe they also pull the results that their "instant answers" are derived from higher up based on the search. If, for example, you did a technical search and their "instant answer" was from a stackoverflow post, that particular post would be pulled higher up the list (but not necessarily to the top) even if it was lower on Bing.

Even Bing doesn’t display the same results that you get from their API. It’s also trivial to boost certain websites and influence rankings that way. But doing that doesn’t change the fact that it’s just a Bing wrapper with some simple boosting and a beautiful UI.

That's a cool site, you should submit it to "Show HN" if you haven't already :)

Not sure if your site is taking the location into account, but when I do the "are vaccines dangerous" search on the search-compare website, I get .gov and .edu domains as the top 5 hits or so.

Glad you like my site. It was fun to build. I remember bing use to have a site years ago where you would search for a term and see bing vs google results and you'd pick the one you preferred, and apparently 2/3 people preferred bing results. But I couldn't find it, so it gave me an idea to build a site but I didn't build the gamified portion.

Yeah, you're right about search results. I'm seeing more normal sites now as well. The results are not deterministic. I'm not sure whats causing it, but I saw different results for the vaccines query earlier. "Flat earth" still gives me a fair amount of flat earth content.

From the FAQ:

> For Duck Duck Go, search-compare searches the html version (e.g. duckduckgo.com/html/?q=search). The results are different than the canonical duckduckgo search, but its easier to scrape. For Google, search-compare uses the main google site (e.g. google.com/search?hl=en&q=search), but it doesn't pass any personally identifiable information

> One thing I found is how bad ddg is at removing conspiracy theories and other misinformation.

Can't be any worse than google since they give preferential treatment to conspiracy theorists like CNN, MBNBC, Foxnews, etc.

> For instance, search "are vaccines dangerous", google replies with .org and .gov sites

So google should just push government and special interest groups funded by the wealthy?

> Even if you search "dangers of vaccines" in google, it still never produces that site.

You make it sound like that's a good thing...

Holy christ man. You are right, let google just show government sites. Big Brother knows best.

> conspiracy theorists like CNN, MBNBC

???

>let google just show government sites. Big Brother knows best.

.org is not a government domain. even so, maybe if they and other tech companies had done this earlier we wouldn't have gotten into this trump + coronavirus mess

> ???

Exactly. This is the problem with the country. Half the crazies believe one set of conspiracies is the truth and the other half believe in another set of conspiracies.

> .org is not a government domain.

Right, I said they were special interests funded by the wealthy.

> maybe if they and other tech companies had done this earlier we wouldn't have gotten into this trump + coronavirus mess

Right... And in a few years, the other half of the crazies will be talking about the biden mess.

Half the morons the last 4 years, Trump stole the election. Half the morons now, Biden stole the election. Figure it out.

That anyone would defend any mainstream media after the last 4 years is being intentionally blind. Right or left.

It's not a conspiracy theory that people are hurt by vaccines. That's a fact. The question is, are people hurt more by the vaccine or the disease itself. From the evidence I've researched, I think it's the vaccines that are more dangerous.

That's interesting. Why don't you publish your research, so it can be peer reviewed?

In case you are serious.

https://en.wikipedia.org/wiki/1976_swine_flu_outbreak

> Three more cases of Guillain-Barré were reported in early December, and the investigation into cases of it spread to eleven states. On December 16, a one-month suspension of the vaccination program was announced by Sencer. William Foege of the CDC estimated that the incidence of Guillain-Barré was four times higher in vaccinated people than in those not receiving the swine flu vaccine.

Yep, my own research also shows that vaccines are dangerous in various ways. Also the power of big pharma and big tech when it comes to money and censorship.

I don’t think we’d be seeing new measles outbreaks if big tech and big pharma were that effective at censoring anti-vaxxers.

You'll find plenty of examples online of cases where the vaccine is responsible for the outbreak of the disease it supposedly protect against.

I can find plenty of examples online of cases where aliens anally probe Dakota farmers. That doesn't mean it's true.

Tip of the ice berg

" LONDON (AP) — The World Health Organization says a new polio outbreak in Sudan is linked to an ongoing vaccine-sparked epidemic in Chad — a week after the U.N. health agency declared the African continent free of the wild polio virus. "

https://apnews.com/619efb65b9eeec5650f011b960a152e9

11 cases is not an epidemic, it's an outbreak.

"a week after the U.N. health agency declared the African continent free of the wild polio virus."

Why do you think the U.N. was able to declare the continent polio free in the first place? Because of strong wishful thinking? Or because of vaccines?

See, that's the thing that enrages me. Here we have an "epidemic" (it's not) that was caused 11 cases of polio - after the virus was eradicated on the continent with the help of the vaccine. An outbreak that wouldn't even be a blip on radar if not for vaccines.

For heaven's sake.