News for hackers. Stuff that matters.
Re:Post Ghost Shutdown: An Open Letter to Twitter by dingo_bat on Saturday July 09, 2016 @01:19AM (#12060223)
Why doesn't PostGhost just record tweets using the browser? I'm sure it can be done without much problem. This is blatant abuse by Twitter. Public tweets are public, and the readers should be able to record/screenshot/save them, without Twitter having a say in the matter.

If they don't allow using the API for that, use the browser directly.

Re:Post Ghost Shutdown: An Open Letter to Twitter by niftich on Saturday July 09, 2016 @01:30AM (#12060260)
Using a 'browser' from their base-of-operations in an automated manner would fit the definition of scraping, which is not allowed by Twitter's TOS.
Re:Post Ghost Shutdown: An Open Letter to Twitter by madeofpalk on Saturday July 09, 2016 @01:37AM (#12060276)
What are they going to do if they break the TOS? It's not like the developer agreement where they'll shut down your API key - the best they can do is try and blacklist your IP Address
Re:Post Ghost Shutdown: An Open Letter to Twitter by mikeryan on Saturday July 09, 2016 @01:43AM (#12060289)
See Craigslist vs 3taps. It's not clear if scraping is legal either.

https://en.wikipedia.org/wiki/Craigslist_Inc._v._3Taps_Inc.

Re:Post Ghost Shutdown: An Open Letter to Twitter by dreamsofdragons on Saturday July 09, 2016 @02:12AM (#12060345)
Wow, google must be on some pretty shaky ground then...
Re:Post Ghost Shutdown: An Open Letter to Twitter by comex on Saturday July 09, 2016 @03:53AM (#12060604)
No, they're not. To quote from the linked Wikipedia article:

> the court held that sending a cease-and-desist letter and enacting an IP address block is sufficient notice of online trespassing, which a plaintiff can use to claim a violation of the Computer Fraud and Abuse Act

If someone wants to block Google, they can simply add a robots.txt entry. An IP block would work too. (Sending a cease-and-desist letter might or might not also work, but it doesn't really matter - context and intent matters in court; Google accepting the standard robots.txt as a 'go away' signal should be more than sufficient.)

Re:Post Ghost Shutdown: An Open Letter to Twitter by joshmn on Saturday July 09, 2016 @02:36AM (#12060415)
Think about The Internet Archive!
Re:Post Ghost Shutdown: An Open Letter to Twitter by sbierwagen on Saturday July 09, 2016 @02:37AM (#12060422)
Google/Alphabet is very close to the Obama administration, so it is unlikely they would be prosecuted: https://theintercept.com/2016/04/22/googles-remarkably-close...
Re:Post Ghost Shutdown: An Open Letter to Twitter by syshum on Saturday July 09, 2016 @04:34AM (#12060680)
In todays shaking legal ground, it would not shock me if Twitter could not get some Agency in the federal government to Arrest the founders/programmers/executives of the site under CFAA and charge them with criminal felonies
Re:Post Ghost Shutdown: An Open Letter to Twitter by rdtsc on Saturday July 09, 2016 @01:41AM (#12060285)
Sue?
Re:Post Ghost Shutdown: An Open Letter to Twitter by bnj on Saturday July 09, 2016 @01:49AM (#12060303)
Scrape using an IP address registered to a corporate body located in an unfriendly jurisdiction.
Re:Post Ghost Shutdown: An Open Letter to Twitter by drdaeman on Saturday July 09, 2016 @01:57AM (#12060316)
Twitter will just block it. Scraping should be done from multiple random addresses from the networks all around the globe, at random intervals. I guess, the only case is voluntary botnet, where users sympathetic to the cause would run limited (so, not prone to abuse) proxies. (To avoid users messing with tweets, just proxy TCP connections to the mothership and let it verify TLS.)

However, this would hinder the service, as with the polling model it may not always notice all the tweets.

Re:Post Ghost Shutdown: An Open Letter to Twitter by joshmn on Saturday July 09, 2016 @02:35AM (#12060412)
luminati.io

Zing.

Re:Post Ghost Shutdown: An Open Letter to Twitter by dottedmag on Saturday July 09, 2016 @08:28AM (#12061092)
Luminati shut downs the accounts if the owners of the scrapped websites complain about it.
Re:Post Ghost Shutdown: An Open Letter to Twitter by downandout on Saturday July 09, 2016 @02:21AM (#12060368)
Perhaps. But it would be up to Twitter to prove such a thing is actually occurring, and it would be difficult or impossible to prove that if setup properly. The information on the site could come from anywhere, since tweets can be viewed even when someone is not logged in.

It is worth noting that TMZ and other sites have been storing and detecting deleted celebrity tweets for years, and still do it to this day. If you're not using the API, Twitter has no affirmative way of knowing how you obtained the information.

Re:Post Ghost Shutdown: An Open Letter to Twitter by mholt on Saturday July 09, 2016 @02:14AM (#12060351)
> "in an automated manner"

I wonder where that line is drawn. A person using their browser, clicking a link to load a web page initiates a hundred different automated routines[1]. Is that not automation? Is it then automation to have a click initiate another click or two more clicks? Where do you draw the line?

[1]: https://github.com/alex/what-happens-when

Re:Post Ghost Shutdown: An Open Letter to Twitter by JeremyBanks on Saturday July 09, 2016 @02:26AM (#12060379)
The spirit of the rule should be self-evident.
Re:Post Ghost Shutdown: An Open Letter to Twitter by ominous on Saturday July 09, 2016 @05:49AM (#12060800)
And yet it isn't.

edit: the spirit of the rule is to grant Twitter control over when to apply the rule. Is that what you meant?

Is [F5] + [CTRL+S](for save) automated? Is a browser auto-refresh capability? is `$ watch wget` (or something to that effect) automated? getpocket.com? an IFTTT recipe? A crawler?

See what happens?

Re:Post Ghost Shutdown: An Open Letter to Twitter by jacquesm on Saturday July 09, 2016 @06:13AM (#12060847)
It is.

Automated means: without human intervention. So anything that accesses more than one URL without human intervention would violate the TOS.

I'm quite amazed that this would have to be spelled out. If you're going to break someone's TOS then at least be aware of it. That helps later on when you have to face the consequences. Just like I'd like to know when I'm speeding, rather than to pretend to be ignorant of the limit. That way I'd know my exposure.

Re:Post Ghost Shutdown: An Open Letter to Twitter by ominous on Saturday July 09, 2016 @06:55AM (#12060917)
It is like you are happy with how soon you truncate "intervention".

Let me know when you find a non human "[F5] + [CTRL+S]". Or link me to the alien crawlers you seem to imply. Are there amazon instances of crawlers owned by non-human entities?

Do you consider the "death by bullet" to be caused by humans? By the transfer of momentum of lead? By gunpowder combustion? By human fingers pressing triggers? By time? It time killing humans? Oh no.

How long must the causal chain be for you to consider it non human?

> [F5] + [CTRL+S]

> a browser auto-refresh capability

> $ watch wget

> getpocket.com

> an IFTTT recipe

> A crawler

I can put all of these in motion. I can also write a macro that triggers them. Is there a difference? Twitter seems to want to be the arbiter of it (if it makes a difference).

That is the only spirit of the law: We reserve the right to consider your actions wrong.

Re:Post Ghost Shutdown: An Open Letter to Twitter by dingo_bat on Saturday July 09, 2016 @03:15AM (#12060514)
So if I just visit twitter without logging in, do I have to abide by their ToS?
Re:Post Ghost Shutdown: An Open Letter to Twitter by 746F7475 on Saturday July 09, 2016 @03:24AM (#12060539)
You may not do any of the following while accessing or using the Services: (i) access, tamper with, or use non-public areas of the Services, Twitter’s computer systems, or the technical delivery systems of Twitter’s providers; (ii) probe, scan, or test the vulnerability of any system or network or breach or circumvent any security or authentication measures; (iii) access or search or attempt to access or search the Services by any means (automated or otherwise) other than through our currently available, published interfaces that are provided by Twitter (and only pursuant to the applicable terms and conditions), unless you have been specifically allowed to do so in a separate agreement with Twitter (NOTE: crawling the Services is permissible if done in accordance with the provisions of the robots.txt file, however, scraping the Services without the prior consent of Twitter is expressly prohibited); (iv) forge any TCP/IP packet header or any part of the header information in any email or posting, or in any way use the Services to send altered, deceptive or false source-identifying information; or (v) interfere with, or disrupt, (or attempt to do so), the access of any user, host or network, including, without limitation, sending a virus, overloading, flooding, spamming, mail-bombing the Services, or by scripting the creation of Content in such a manner as to interfere with or create an undue burden on the Services.

Relevant part: "You may not do any of the following while accessing or using the Services: (...) crawling the Services is permissible if done in accordance with the provisions of the robots.txt file, however, scraping the Services without the prior consent of Twitter is expressly prohibited"

Re:Post Ghost Shutdown: An Open Letter to Twitter by dingo_bat on Saturday July 09, 2016 @04:52AM (#12060715)
But the thing is, I didn't read or agree to the ToS at all. It was never presented to me and content was presented on my screen when I visited twiiter.com, or followed a link. I am not bound by terms I did not agree to.

At least that is my layman logic. But we live in a world where ~50 page long EULAs exist, so layman logic may not be applicable in this case.

Re:Post Ghost Shutdown: An Open Letter to Twitter by heroprotagonist on Saturday July 09, 2016 @06:46AM (#12060904)
True. Consider also whether adding something like "Each request to this server constitutes agreement to a $50 fee which is selectively enforced at our discretion" would make a scraper liable when they didn't agree to the terms.

If the law truly agrees that acceptance of undisclosed terms is granted simply by virtue of access, it opens the door to rampant abuse.

Though, if you want to try the protecting yourself in a similar approach, add a header to every request in your browser that says "By responding to this request, you accept that I do NOT agree to your terms and you are willing to serve me anyway." At least in this case they would actually receive your terms with the request. I wish there was an RFC for this.

Re:Post Ghost Shutdown: An Open Letter to Twitter by mikeryan on Saturday July 09, 2016 @04:56PM (#12063285)
The way this works isn't that you do something against the TOS and then they try to sue you out of existence. The way it will work is that Twitter will notify you "Hey what you're doing is against our TOS". At which point you'll be aware and if you continue your actions you're now liable.
Re:Post Ghost Shutdown: An Open Letter to Twitter by 746F7475 on Saturday July 09, 2016 @03:07PM (#12062795)
I'm guessing the word to pay attention to is "accessing". So I guess as long as you are accessing the site you are under ToS.
Re:Post Ghost Shutdown: An Open Letter to Twitter by 010a on Saturday July 09, 2016 @02:20AM (#12060365)
Which doesn't matter if you are based in a country that doesn't respects US law.
Re:Post Ghost Shutdown: An Open Letter to Twitter by syshum on Saturday July 09, 2016 @04:37AM (#12060688)
If your nation has a extradition Treaty with the US I would be cautious as well...

US has been very good of late about charging people that have never been to the US for violation of US laws online. It is bullshit but it seems many nations refuse to protect their citizens from harassment by the US legal system

Re:Post Ghost Shutdown: An Open Letter to Twitter by jonknee on Saturday July 09, 2016 @01:23AM (#12060240)
Copyright issues don't go away if you capture via browser.
Re:Post Ghost Shutdown: An Open Letter to Twitter by JoshTriplett on Saturday July 09, 2016 @01:29AM (#12060253)
Copyright belongs to the person tweeting, not to Twitter (they just get a license from the person tweeting). And fair use specifically mentions "criticism, comment, news reporting, [...] scholarship, or research", several of which apply to reporting on deleted tweets from prominent public figures.

This isn't a copyright issue at all; it's exclusively a ToS issue. Twitter can choose to prevent someone from using their API. They'd have a harder time forcing the takedown of the existing content.

Re:Post Ghost Shutdown: An Open Letter to Twitter by simbalion on Saturday July 09, 2016 @03:49AM (#12060595)
It's a 1st amendment issue. If someone says something in a public forum what they said becomes public domain.
Re:Post Ghost Shutdown: An Open Letter to Twitter by jonknee on Saturday July 09, 2016 @01:46AM (#12060296)
Yes, and what do you think of the person who deleted a tweet will say about you using their copyright? I would argue it is a copyright issue because by deleting content a user is saying they don't want anyone to access it. Why would Twitter then want to turn around and allow access to it through an API?

As a user I think this is the right tack to take. There is no possible advantage for Twitter to have sites that make the delete feature not really work.

Re:Post Ghost Shutdown: An Open Letter to Twitter by wpietri on Saturday July 09, 2016 @02:32AM (#12060401)
Copyright is the right we give to content creators so that they can financially benefit from their work. It is not a generic "I control everything I have ever said" power.

In particular, "fair use" specifically doesn't require people to get permission from a copyright holder:

http://fairuse.stanford.edu/overview/fair-use/what-is-fair-u...

Discussing the tweets of public figures is clearly "commentary and criticism", which would give things like PostGhost "substantial non-infringing use" (which I believe is the relevant legal threshold).

Re:Post Ghost Shutdown: An Open Letter to Twitter by thaumasiotes on Saturday July 09, 2016 @02:53AM (#12060462)
> and what do you think of the person who deleted a tweet will say about you using their copyright?

How could this possibly be relevant? Copyright won't give them any grounds to stop you from saying "he's trying to walk it back now, but he said <embarrassing statement> earlier".

Re:Post Ghost Shutdown: An Open Letter to Twitter by Dylan16807 on Saturday July 09, 2016 @01:52AM (#12060306)
> Why would Twitter then want to turn around and allow access to it through an API?

Then? No, this is about getting it through the API while it's still up.

Re:Post Ghost Shutdown: An Open Letter to Twitter by jonknee on Saturday July 09, 2016 @01:59AM (#12060320)
Obviously, but with the restriction you also delete it when it's no longer in the API. It would be ridiculous if Twitter helped developers to embarrass/blackmail their users.
Re:Post Ghost Shutdown: An Open Letter to Twitter by cyphar on Saturday July 09, 2016 @02:55AM (#12060470)
If it is stored for purposes of commentary or news reporting, then it would fall under fair use and neither Twitter nor the original author have any power to stop you from republishing the tweets as long as it is considered fair use.
Re:Post Ghost Shutdown: An Open Letter to Twitter by Dylan16807 on Saturday July 09, 2016 @03:31AM (#12060560)
You can't undo saying something to a hundred thousand people. You can demand everyone forget, but it won't work.
Re:Post Ghost Shutdown: An Open Letter to Twitter by edoceo on Saturday July 09, 2016 @11:14AM (#12061584)
Big Brother, 1984
Re:Post Ghost Shutdown: An Open Letter to Twitter by roywiggins on Saturday July 09, 2016 @02:00AM (#12060321)
When someone's an verified user, they're a public figure and have an outsize impact on the Twitter discourse (they tend to have a lot of followers). The things they say are inherently newsworthy. If the archiving is done for the purpose of scholarship or commentary it sounds like a compelling argument for it being fair use.
Re:Post Ghost Shutdown: An Open Letter to Twitter by thwarted on Saturday July 09, 2016 @02:15AM (#12060355)
I think that one of the things that an account should have to trade in for being a Verified Account is the ability to delete tweets. Being a public figure and having the notoriety that is implicit in having a Verified Account, the account should have to stand by what it tweets, or at least not be able to modify the public record and break references to its content.
Re:Post Ghost Shutdown: An Open Letter to Twitter by exolymph on Saturday July 09, 2016 @02:58AM (#12060479)
I like this idea, but I think a big function of the checkmark is reassuring people that an account belong to who it says it belongs to. For example, @realDenaldTrump is not verified (even though people still fall for it).

That said, Twitter administers the checkmark in puzzling and thoroughly non-transparent ways.

Re:Post Ghost Shutdown: An Open Letter to Twitter by thwarted on Saturday July 09, 2016 @01:19PM (#12062324)
I think a big function of the checkmark is reassuring people that an account belong to who it says it belongs to

That is in fact the only reason Verified Accounts exist, and is the reason that non-celebrities or people otherwise lacking notoriety can not get Verified Accounts. It's not worth Twitter's effort to make my account, or any random user, Verified, it seems. But as Twitter's role continues to expand in the public sphere, it is really the public who gives out Verified Accounts (through making someone famous), and the public should have requirements, such as a certain level of consistency, from the accounts it gives the power/influence of the Verified Account to.

Re:Post Ghost Shutdown: An Open Letter to Twitter by homero on Saturday July 09, 2016 @01:26AM (#12060247)
Then do it anonymously