News for hackers. Stuff that matters.

A comment on Google No Longer Automatically Indexes Websites – WTF?. Parent comment

Re:Google No Longer Automatically Indexes Websites – WTF? by jeffbee on Monday June 05, 2023 @11:19AM (#36197482)
> so I know it's crawled that page, it often doesn't return that page either.

This is your misunderstanding. The fact that a thing was in the index does not ensure it will always be there. Things disappear from the web all the time. Serving fresh docs means not only crawling the new stuff but also deleting the unreachable stuff promptly.

[ Parent | on HN ]
Re:Google No Longer Automatically Indexes Websites – WTF? by userbinator on Monday June 05, 2023 @07:57PM (#36206122)
No, I literally did a site: search seconds after visiting the site to see if it had any other pages with what I was looking for, and it found zero results --- not even the original page I found.
[ Parent | on HN ]
Re:Google No Longer Automatically Indexes Websites – WTF? by jeffbee on Monday June 05, 2023 @09:15PM (#36206808)
And you simply refuse to believe that online index updates exist?
[ Parent | on HN ]