Re:How Google’s Web Crawler Bypasses Paywalls
by jsprogrammer on Friday February 19, 2016 @04:55PM
(#11136937)
So...if I only entitle Mozilla based agents to view my content and Chrome, IE, and Safari users still get access by claiming to be a Mozilla agent, those users have committed a felony?
This line of argumentation is beyond ridiculous.
Re:How Google’s Web Crawler Bypasses Paywalls
by Spare_account on Friday February 19, 2016 @05:15PM
(#11137078)
Are your Mozilla users paying for access? Are the Chrome users with the fake agent strings obtaining paid-for content for free? The use of forged agent strings isn't the offence.
Re:How Google’s Web Crawler Bypasses Paywalls
by oldmanjay on Friday February 19, 2016 @03:48PM
(#11136505)
I wasn't playing a linguistic game. You mentioned standards and malformed requests, and I pointed out that you misused those terms. I am not bothering to track down definitions for you to play, as you say, linguistic games. You are free to find something that proves my position wrong, as I used the RFC to cite my position as correct.
If you meant to put forth that faking a user agent is a technique to exceed authorization, that's fine, and I'm glad to have helped you clarify it. Just be clear, it's not what you said with your detour into malformed requests.
Obtaining paywall-protected content by faking your user agent to purport yourself to be a Google Crawler is quite clearly fraudulent. This isn't a point for debate.
PS. To play along with the linguistic theme, can you provide a source for the definition of a malformed request? My original intent when using the word malformed was not to invoke it's technical definition but rather it's dictionary definition. But, having said that, I just had a 30 second Google hunt and couldn't find anything to corroborate your position.