News for hackers. Stuff that matters.
Re:How Google’s Web Crawler Bypasses Paywalls by profeta on Friday February 19, 2016 @02:36PM (#11135976)
intent can be many things.

i disable javascript. so i can't even see their page (and hence i don't know my access is being denied since i got a 200 http response, which means "OK") so i try different user agents with the intent of reading the content they are providing. Just like microsoft case.

There are no comments on this comment yet.