News for hackers. Stuff that matters.

A comment on Twitter now requires an account to view tweets. Parent comment

Re:Twitter now requires an account to view tweets by raincole on Saturday July 01, 2023 @03:34AM (#36547681)
If Google doesn't provide a way to say "please don't scrape my site", then it 100% unethical.

We have robots.txt. If Google doesn't respect that, it's unethical. Don't you think so?

[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by pcthrowaway on Saturday July 01, 2023 @05:41AM (#36548388)
Maybe it's time for an llm.txt

Not that the people you want to respect that would

[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by raincole on Saturday July 01, 2023 @06:23AM (#36548583)
The tricky part is it's much more harder to prove that they didn't respect that.
[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by RobotToaster on Saturday July 01, 2023 @06:32AM (#36548645)
Does twitter's robots.txt forbid scraping? Judging by the fact it shows up in Google I'd assume not.
[ Parent | on HN ]