News for hackers. Stuff that matters.
Re:How Google’s Web Crawler Bypasses Paywalls by mattmaroon on Friday February 19, 2016 @02:57PM (#11136137)
Yep. I then pasted it but it didn't work on wsj.com. Oh well.
Re:How Google’s Web Crawler Bypasses Paywalls by elaineo on Friday February 19, 2016 @03:10PM (#11136240)
try deleting cookies, then hit refresh.
Re:How Google’s Web Crawler Bypasses Paywalls by mrgrieves on Friday February 19, 2016 @05:15PM (#11137079)
Not working here either. I tried wiping out cookies and cache, and Chrome is sending the right user agent (Googlebot).
Re:How Google’s Web Crawler Bypasses Paywalls by nzealand on Saturday February 20, 2016 @01:34AM (#11139179)
Try disabling other extensions that might alter the user agent.