News for hackers. Stuff that matters.

A comment on Twitter cut off the ability to read a tweet by fetching its URL with a HTTP GET. Parent comment

Re:Twitter cut off the ability to read a tweet by fetching i... by sethaurus on Friday December 18, 2020 @04:52AM (#25465530)
It’s been received wisdom until now that Google penalizes websites which behave differently when scraped by the Googlebot. Is that no longer the case?
[ Parent | on HN ]
Re:Twitter cut off the ability to read a tweet by fetching i... by SXX on Friday December 18, 2020 @05:19AM (#25465697)
Pinterest has proven by spamming SERP for years that if you're big enough Google will close eyes on it.
[ Parent | on HN ]
Re:Twitter cut off the ability to read a tweet by fetching i... by syshum on Friday December 18, 2020 @06:09AM (#25465958)
That applies to more than just google

If you are big enough, there are separate rules for you (or no rules)

[ Parent | on HN ]
Re:Twitter cut off the ability to read a tweet by fetching i... by smarx007 on Friday December 18, 2020 @06:42AM (#25466139)
They will serve same content to users with JS enabled and legit Googlebots while blocking clients w/o JS and bots. I don't think it violates Google rules but ofc is of questionable decency.
[ Parent | on HN ]
Re:Twitter cut off the ability to read a tweet by fetching i... by ec109685 on Friday December 18, 2020 @09:29AM (#25467394)
And Google can verify by crawling the site with and without JavaScript.

Also, google has a commercial relationship with them: https://www.convinceandconvert.com/social-media-research/twi...

[ Parent | on HN ]