Google No Longer Automatically Indexes Websites

Re:Google No Longer Automatically Indexes Websites – WTF? by ballenf on Monday June 05, 2023 @12:52PM (#36199191)

Also on the HN front page right now is an example of the price we pay when we prune "undesirable" websites from our search indices:

https://www.quantamagazine.org/sci-fi-writer-greg-egan-and-a...

I don't know if 4chan is included in the google index, but I've never gotten a 4chan result in any search I can recall.

[ Parent | on HN ]

Re:Google No Longer Automatically Indexes Websites – WTF? by tayo42 on Monday June 05, 2023 @01:04PM (#36199379)

why would you expect to get a 4chan page? none of that data is persistent? iirc google relies on links to the page, so that is impossible, plus the content rotates constantly when they drop off the last page

[ Parent | on HN ]

Re:Google No Longer Automatically Indexes Websites – WTF? by ipaddr on Monday June 05, 2023 @01:54PM (#36200213)

Because you get 4chan results in duckduckgo and yandex.

[ Parent | on HN ]

Re:Google No Longer Automatically Indexes Websites – WTF? by tayo42 on Monday June 05, 2023 @02:09PM (#36200460)

What's the query and result?

I still don't get how you expect it to work when the content rotates quickly and disappears

If you search 4 Chan on Google it does come up in the results and a safe search warning

[ Parent | on HN ]

Re:Google No Longer Automatically Indexes Websites – WTF? by chownie on Monday June 05, 2023 @03:46PM (#36202462)

Only a couple boards on 4chan update quickly, the vast majority contain threads which stand for months at a time.

[ Parent | on HN ]

Re:Google No Longer Automatically Indexes Websites – WTF? by jstarfish on Monday June 05, 2023 @02:56PM (#36201334)

Funny you say that. I got referred to the local AI models thread (/lmg/) on the technology board just the other day.

[ Parent | on HN ]