I guess I’m surprised that it’s not possible to quickly deindex these spam farms and render their SEO spend a waste. Especially Google, I’m surprised they can’t detect and ban SEO rings more effectively.

I am surprised by this as well since Google is able to identify the origin of content (eg. Stackoverflow) and penalize sites for duplicating that content (or so they say). In my opinion it wouldn’t be difficult to identify the domains duplicating the content and penalize heavily. But, those sites are most likely displaying Google Ads so…

I would assume this is easier if you are indexing more frequently and have been indexing for longer, and Google has more resources to allow them to crawl more often (another advantage to scale in search, which combined with other advantages of scale, is why Google has an effective monopoly IMO).

If you recently started crawling you won't be able to detect which content was 'first', and if you don't crawl as frequently as google you are reliant on your crawl frequency being quicker than the 'content stealing' speed.

> if you don't crawl as frequently as google you are reliant on your crawl frequency being quicker than the 'content stealing' speed.

No problem there. If a page hasn't been crawled, it won't be a search result.

I mean you need to crawl often enough that you can determine which site stole content.

If a full crawl of the web takes 3 months and A steals content from B after 2 weeks, it may not be possible to easily infer that A has stolen from B rather than visa-versa.

If I crawl every week and see that A posted it on the first crawl, and B suddenly also had that content on the second crawl, I can infer that B stole from A.

I just wish I could configure google to not show certain domains such as W3schools without adding a complex search query or installing plugins

To be fair, w3schools and the like don't just copy content. They provide different information and, most importantly, present it differently from official docs. And for many, their way of presenting information may work better.

I acknowledge your opinion but still do not want to see that and several other spam sites over more official sources such as mdn.

The W3schools hate is a bit outdated. Even W3Fools[1] eventually backtracked and acknowledged that W3Schools has continuously improved and updated their content.

1. https://www.w3fools.com/

This is such an easy optimization that Google has to have tried it and seen their ad revenue fall.

"Permanently remove this site from search results" seems like the absolute easiest "personalization" to implement.

And the most certain too. You don't need to guess whether or not the user really needs that result

But what will Google do if users end up banning the majority of websites with Google Ads and/or analytics?

The reason Google doesn't fight SEO spam (and just generally annoying/obnoxious behavior such as paywalls) more aggressively is because it doesn't want to bite the hand that feeds - all these spammy sites have ads or analytics that benefit Google.

Maybe the good folks at Apple, Mozilla, Brave or Opera could implement this feature in their browsers? They’re looking to increase their percent of browser use - it could encourage people to at least try switching over. Obviously would be popular among the HN crowd for developer documentation, but we could spread the news :)

On the downside I could see this being used by terrible people to delist actual information sources so that they would only ever be served conspiracy theory sites in their searches as well.

I don't think the first two companies you've mentioned care about making search better, as they're both getting a fat check from Google.

Google did it with a lot of health blog spam that had low quality and false info but a ton of referral links. It takes a lot of bad publicity for them to act, though.

The SEO places aren't dumb, they do their best to mimic legit content.

True. If you search for any Python documentation the first links on Google are almost always geeksforgeeks, w3schools, etc instead of the official Python docs.

I think this is telling of how often the python docs actually solve peoples’ questions. For example, I searched “python string replace”, with the first 4 being tutorialpoint, w3schools, geeksforgeeks, stack overflow and at position 8 is the python docs.

The issue with the python docs for this search is that it takes me to “built-in types”, and starts off with things totally irrelevant to what I want. I now have to go searching for string, then find the built-in method replace. Meanwhile, the other tutorial-esque websites almost all show an example of str.replace above the fold. If you’re Google and you’re trying to get people information faster, you’re going to promote the result that has more people not coming back to the search result page afterwards because it was hard to find the answer on that result.

I think string replacement in Python is a particularly egregious example because strings are core language types. Library docs are a lot better organized and a lot more easily indexed. This is just a problem with Python's docs in general, imo.

You want Google to direct you to the correct anchor on the page. That makes sense, Google should do that. The docs shouldn't be broken out to 1 page per method because Google likes to link to the top of the page instead of the relevant anchor tag.

This is true, and if you want the docs you can always just add the word docs to your query. Expecting the search engine to always favor one thing seems like it'd work well in this case for some people, but in other cases might get mushy

I personally configure custom search engines for things like this so if I want docs specifically, I'll type something like "py collections" which directs me to https://docs.python.org/3.10/search.html?q=collections.

I find this especially useful since I often otherwise find myself typing site:python.org (or similar things, like if I just want to see the MDN page instead of going through w3schools or blog posts for examples).

edit: in the particular example for str.replace, I found it buried in the results for "py replace", at the top of "py str.replace" (helps if you know what Python calls the type) and basically nowhere for "py string replace".

Yup, python docs need to look at cppreference

You should try adding "docs" to the query. "range python" shows w3schools at the top for me but "range python docs" shows docs.python.org first for me. Same for "html article tag" (w3schools) vs "html article tag docs" (MDN).