News for hackers. Stuff that matters.

A comment on Twitter now requires an account to view tweets. Parent comment

Re:Twitter now requires an account to view tweets by bojo on Saturday July 01, 2023 @02:42AM (#36547368)
So, are we saying it's unethical for Google and other search engines who make money off of ad revenue to scrape sites like Twitter? Or are they paying a large sum to Twitter to do this?
[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by cygx on Saturday July 01, 2023 @03:39AM (#36547721)
If done against the wishes of the owner of the site, yes, I would consider that unethical. Thankfully, Google respects robots.txt and noindex.
[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by zladuric on Saturday July 01, 2023 @07:36AM (#36549024)
But it's it ethical for the site owner to block access to random people and companies in the internet to _my_ data? I posted that tweet with the expectation that it's gonna be publicly available. Now the owner of the site is breaking that expectation. I would say that this part is also unethical.

Especially since they're not moderating things or anything.

[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by cygx on Saturday July 01, 2023 @07:51AM (#36549121)
I would say that this part is also unethical.

Agreed. However, it's probably covered by their terms of service.

Same thing with the recent reddit kerfuffle. I'd have much preferred a Usenet 2.0 instead of centralizing global communications in the hands of a handful of private companies with associated user-hostile incentive structures.

[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by spullara on Saturday July 01, 2023 @02:55AM (#36547446)
When there is a value exchange between the two entities that are relatively similar then I think it is ethical. People trade Google making money on ads for their site being found when people search. It is also possible to opt-out.
[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by raincole on Saturday July 01, 2023 @03:34AM (#36547681)
If Google doesn't provide a way to say "please don't scrape my site", then it 100% unethical.

We have robots.txt. If Google doesn't respect that, it's unethical. Don't you think so?

[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by pcthrowaway on Saturday July 01, 2023 @05:41AM (#36548388)
Maybe it's time for an llm.txt

Not that the people you want to respect that would

[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by raincole on Saturday July 01, 2023 @06:23AM (#36548583)
The tricky part is it's much more harder to prove that they didn't respect that.
[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by RobotToaster on Saturday July 01, 2023 @06:32AM (#36548645)
Does twitter's robots.txt forbid scraping? Judging by the fact it shows up in Google I'd assume not.
[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by wiseowise on Saturday July 01, 2023 @02:59AM (#36547474)
They benefit mutually from their symbiosis. Financially, AI bro model #1321 doesn’t bring anyone value except their owners.
[ Parent | on HN ]
Re:Twitter now requires an account to view tweets by max51 on Saturday July 01, 2023 @03:02AM (#36547491)
Being indexed by google is optional. Twitter could stop it a any time if they thought it was a bad deal for them. That not comparable to a startup company trying to scrape the entire site to train their AI and using sophisticated techniques to bypass protections Twitter has put in place
[ Parent | on HN ]