News for hackers. Stuff that matters.
Re:How Google’s Web Crawler Bypasses Paywalls by slig on Friday February 19, 2016 @01:22PM (#11135348)
If they're now blocking clicks from Google, doesn't that mean that they're cloaking and violating the Google's Webmaster Guidelines [1]?

[1]: https://support.google.com/webmasters/answer/66355?hl=en

Re:How Google’s Web Crawler Bypasses Paywalls by elaineo on Friday February 19, 2016 @01:25PM (#11135387)
Google is not okay with cloaking, but they will whitelist publishers if the publisher specifically includes a parameter that declares if the site requires registration or subscription. This is done in the sitemap.

https://support.google.com/news/publisher/answer/74288?hl=en

Re:How Google’s Web Crawler Bypasses Paywalls by ikeboy on Friday February 19, 2016 @02:25PM (#11135881)
https://support.google.com/news/publisher/answer/40543?hl=en seems to specifically ban this. WSJ is in violation, not fitting any of the categories there.
Re:How Google’s Web Crawler Bypasses Paywalls by slig on Friday February 19, 2016 @01:29PM (#11135417)
I didn't know that, thanks. But reading, it seems that option is about Google News, not the main Google Search.

> News-specific tag definitions

> Yes, if access is not open, else should be omitted

> Possible values include "Subscription" or "Registration", describing the accessibility of the article. If the article is accessible to Google News readers without a registration or subscription, this tag should be omitted.

Re:How Google’s Web Crawler Bypasses Paywalls by pacquiao882 on Friday February 19, 2016 @03:58PM (#11136566)
WSJ is big enough to negotiate their own terms with Google Search.
Re:How Google’s Web Crawler Bypasses Paywalls by morgante on Friday February 19, 2016 @07:27PM (#11137854)
They're really not. They need Google a lot more than Google needs them.

One of these things is going to happen:

(1) They end this "experiment."

(2) They stop serving Google the full content. (And see their rankings drop accordingly.)

(3) They get delisted for cloaking.

Re:How Google’s Web Crawler Bypasses Paywalls by CM30 on Friday February 19, 2016 @04:07PM (#11136630)
Yes, and I suspect that's what going to get them to change it back again. After a ban from Google and the traffic drop it'll likely bring, that paywall is likely coming right back down. An awful lot of media companies made similar mistakes before, and it's always ended with them quickly removing their 'work arounds'.
Re:How Google’s Web Crawler Bypasses Paywalls by Implicated on Friday February 19, 2016 @07:35PM (#11137917)
Yea, except they're not going to get that ban.