Re:How Google’s Web Crawler Bypasses Paywalls
by ikeboy on Friday February 19, 2016 @02:32PM
(#11135942)
New workaround: paste the article title into archive.is. I don't know what they're doing but they have a workaround of some sort.
[ Parent
| on HN ]
Re:How Google’s Web Crawler Bypasses Paywalls
by mikedmiked on Friday February 19, 2016 @05:22PM
(#11137126)
This is what I do.
I actually made a bookmarklet with the following pasted into the URL, so you can do it in a single click:
javascript:void(open('https://archive.is/?run=1&url='+encodeURIComponent(document....)
Re:How Google’s Web Crawler Bypasses Paywalls
by LunaSea on Friday February 19, 2016 @07:14PM
(#11137789)
That's actually really interesting!
Could anyone chime in and explain how they might work around this issue?
Re:How Google’s Web Crawler Bypasses Paywalls
by ikeboy on Friday February 19, 2016 @07:22PM
(#11137837)
My guess is they have a login, but they could just be using some workaround like described in OP.
I've used them to save Facebook posts before, and the pages were logged in to some "Nathan" IIRC. They probably have a bunch of hacks for specific sites that needed fixing.