WSJ Ends Google Users' Free Ride, Then Fades in Search Results

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by devrandomguy on Monday June 05, 2017 @05:07PM (#14492060)

Is that really a thing? That must be such a hazard for their developers. I usually have a test for sites that I work on, that scrapes a few URLs as Googlebot, to verify that they are getting an optimized view (no JS, structural-only css).

[ Parent | on HN ]

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by Normal_gaussian on Monday June 05, 2017 @05:34PM (#14492328)

God. That's the reason so many sites look great in the results and are confusing interactive messes when I get into them.

[ Parent | on HN ]

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by devrandomguy on Monday June 05, 2017 @05:37PM (#14492363)

If it's any consolation, the site is well tested in Noscript mode.

[ Parent | on HN ]

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by SomeCallMeTim on Monday June 05, 2017 @06:01PM (#14492605)

You sir are a god among Web developers.

If only they all did this. So many sites I get to and they're a blank page or an absolute disaster....

[ Parent | on HN ]

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by ChuckMcM on Monday June 05, 2017 @05:36PM (#14492349)

Yes. Googlebot only crawls from legit addresses (even when their developers are trying new things) so it's an easy scraper/scammer signal to key off of.

[ Parent | on HN ]

Re:WSJ Ends Google Users' Free Ride, Then Fades in Search Re... by dmn001 on Monday June 05, 2017 @06:03PM (#14492623)

No. Most websites don't do this.

[ Parent | on HN ]