Re:Cloudflare, Google, Bing Destroying the Infrastructure of...
by greatjack613 on Wednesday August 26, 2020 @07:46PM
(#24288613)
The solution is simple. Can some cloudflare engineers here grant this search engine the same access it gives the chinese ones?
[ Parent
| on HN ]
Re:Cloudflare, Google, Bing Destroying the Infrastructure of...
by zerotolerance on Wednesday August 26, 2020 @07:49PM
(#24288633)
They failed to present data that exceptions are made for other search engines.
Re:Cloudflare, Google, Bing Destroying the Infrastructure of...
by greatjack613 on Wednesday August 26, 2020 @07:52PM
(#24288659)
Valid point, but I do not think they are lying, I am sure they can present it to cloudflare if required.
Re:Cloudflare, Google, Bing Destroying the Infrastructure of...
by joshuamorton on Wednesday August 26, 2020 @08:17PM
(#24288826)
It's quite likely that Baidu is doing the reasonable thing and using a more sophisticated crawling mechanism to cache the cookies. You have to do this anyway to be able to crawl much of the web which is highly js-dependent.