Re:Google No Longer Automatically Indexes Websites – WTF?
by HWR_14 on Monday June 05, 2023 @12:34PM
(#36198845)
The web indexes from Google and Bing are available publicly? I can pull it down from somewhere and try to make a search engine?
[ Parent
| on HN ]
Re:Google No Longer Automatically Indexes Websites – WTF?
by pixodaros on Monday June 05, 2023 @01:01PM
(#36199342)
Bing at least license their indexes to partners on a commercial basis, as did Yahoo until they gave up indexing the web. I am sure that the NSA, the Chinese government, the ahrefs website, and other organizations have comprehensive indexes of the web which they don't share in this way.
Mojeek seems to be the independent, non-paywalled search engine with the biggest index, for an overview see https://seirdy.one/posts/2021/03/10/search-engines-with-own-...