Which doesn't matter, because the WSJ is never going to sue you. But make sure you only consider your justification a personal one, not one that would provide any legal protection.
All I want to say is scrape cfaa now!
pun?
i disable javascript. so i can't even see their page (and hence i don't know my access is being denied since i got a 200 http response, which means "OK") so i try different user agents with the intent of reading the content they are providing. Just like microsoft case.
It is not specifically defined in the law, so it reverts to the traditional meaning: anything the owner of the system says you aren't authorized to access.
It's lunacy, I know. That's what HTTP headers and WAFs and such are for. But that's the stupid law, and it sent someone who used to be my friend to federal prison for changing a user agent and referrer and accessing unprotected data on the web.
Tread carefully.
Knowingly and with intent to defraud, accesses a
protected computer without authorization, or exceeds
authorized access, and by means of such conduct
furthers the intended fraud and obtains anything of value,
The courts have interpreted "protected computer" as any computer connected to the internet.The mismatch between the world views of jurisprudence and engineers is a neverending source of joy. (If working in tech has made you cynical like me, that is.)
- to access a computer with authorization and to use such access
to obtain or alter information in the computer that the accesser
is not entitled so to obtain or alter.
So arguing User-agent is not an authorization mechanism probably won't help you, because exceeding authorization means, first, that you were authorized to access the computer (HTTP GET returns 200) but then that you used that access to obtain information in the computer that you were "not entitled so to obtain."Again, exceeding authorized access means using your authorized access to obtain information you were not "entitled" to. So the question is not 'were you authorized' but rather it is 'were you entitled' to that information? WTF 'entitled' means is another question entirely, but likely it is in the eye of the beholder. A jury decided Weev was not 'entitled' to the email addresses he downloaded from AT&T, and it's safe to assume we are not 'entitled' to free access to WSJ's content. So I would not rest your hopes on the "200 OK".
So, your example is...not an example?
>WTF 'entitled' means is another question entirely
No, in this case it is very clear: a request containing a particular user agent string is entitled. I have not tried this myself, but presumably you could verify that is the case by sending a request with the appropriate user agent.
Again I think you're confusing the fact someone could trick the server into delivering the content for free with WSJ intending to deliver their content to you for free. Since WSJ clearly intends their content to be delivered to only Googlebot for free and to users only if they pay, it is likely a jury would consider this a violation of CFAA.
A web server returning 200 OK is not ipso facto a guarantee the person making the request is not committing a crime. To give a more obvious example, if the request header contains a stolen authorization token. The law does not require the access control be non-trivial to defeat.
I don't like it, and I think the CFAA is seriously problematic, but it is the law and the Feds have been known to enforce it.
It is not at all clear that WSJ intends Googlebot to get their content for free while others must pay. Thus is actually against Google's policies, which would actually call into question whether WSJ's behavior is felonius. WSJ may not be entitled to be incorporated into Google's index, yet they are manipulating the Googlebot to the contrary.
At least in the US, the law doesn't work that way. Decisions will quite often cite some other similar case which reached the opposite conclusion, but under different circumstances, because that other case's decision says something like "X, if it weren't for Y" or "Fortunately for the defendant, they didn't Z, so not X", or something. That isn't binding precedent for the judge to apply X, but it's a very strong sign that X would be reasonable.
A court case that says "Yes, this violates CFAA but we have to throw out the case because A, B, and C" is very strong reason to believe that, if the next prosecutors avoid A, B, and C, the next judge will say "Yes, this still violates CFAA."
(IANAL but I read court cases because I find it useful to understand my jurisdiction's legal system.)
> a request containing a particular user agent string is entitled.
The phrasing of the law is very clear that the word "entitled" applies to a person, not to a request. Stealing someone's password and using their account is definitely a violation of CFAA (see e.g. http://www.wiggin.com/16332). In such a case, the account used to log in is quite plainly "entitled" / "authorized;" that's how you get the data. But the person logging in is not "entitled".
The other decision was vacated. The jury was not appropriate and their decision is irrelevant.
Unfortunately, when the prosecutors come, they will only care about the legal process.
The legal world cares about how the law applies to the facts of the case, not about how common sense applies.
Not saying I like it.
However, the set of people "authorized" is not, at least not from a legal perspective. This is what the case law says. The fact that the set of people who technically _can_ access the data is different from the set of people legally authorized to access the data.
That might not be what the engineers who designed the system, run it, and produce the content intended, but that is what the law says.
It's a bummer the two disagree. But only one of the two systems put you in jail if you cross them.
You and I may wish it were otherwise, but wishing isn't going to make it so.
> Likewise, implementations are encouraged not to use the product tokens of other implementations in order to declare compatibility with them, as this circumvents the purpose of the field. If a user agent masquerades as a different user agent, recipients can assume that the user intentionally desires to see responses tailored for that identified user agent, even if they might not work as well for the actual user agent being used.
That sure sounds like impersonating other user agents is allowed, but not encouraged. That is a clear distinction from being malformed.
Obtaining paywall-protected content by faking your user agent to purport yourself to be a Google Crawler is quite clearly fraudulent. This isn't a point for debate.
PS. To play along with the linguistic theme, can you provide a source for the definition of a malformed request? My original intent when using the word malformed was not to invoke it's technical definition but rather it's dictionary definition. But, having said that, I just had a 30 second Google hunt and couldn't find anything to corroborate your position.
This line of argumentation is beyond ridiculous.
If you meant to put forth that faking a user agent is a technique to exceed authorization, that's fine, and I'm glad to have helped you clarify it. Just be clear, it's not what you said with your detour into malformed requests.
That is demonstrably false. I, personally, can consider any request malformed unless it starts with the letter W. Regardless of what the standards say, I can think whatever I want to.
Similarly, the law can make up whatever rules IT wants to about the definition of "malformed". In that case, it pays some scant attention to things like standards, but mostly cares about "to the random guy-on-the-street (jury member or judge) did this seem like stealing". And there, I am afraid you lose.
I could say "the sky is blue" and you could say "demonstrably false; the sky is personally red because I'm wearing colour-altering sunglasses".
What he meant is obviously "a request can only be considered malformed (as defined by the commonly accepted definition of malformed) .."
Your argument is fallacious. The courts will use common definitions of terms and have regard for context.
However, what you're arguing about is even more pointless. The CFAA doesn't depend on the term "malformed" in any way, but on the term "authorized access".
Your first sentence is not similar at all to your second; they're demonstrating first a blatant disregard for common definitions of terms, followed by praise of such common definitions.
If you don't get it, court doesn't care about what's reasonableness, technically correctness, etc. Only if your lawyers can convince jury/judge. Twinky made me crazy, It the gloves don't fit... and so forth.
"Here is a transcript of electronic forum detailing how to circumvent access controls and defraud the victim using the exact methods defendant used to access victim's website. A forum the defendent heavily traffics. Often multiple times per day."
Ladies and gentlmen I ask you is it more likely that the defendent, a self professed developer, and expert in these circumvention methods, who reguralry participates on forums discussing hacking and defrauding companies such as the victim. I ask you is it reasonable to believe he "just forgot"?
Lawyers man, Lawyers! Can you not understand that rationality, technicallity don't matter. Lawyering is like statistics/graphs. You can get the data to say whatever you want.
general.useragent.override.netflix.com;Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:42.0) Gecko/20100101 Firefox/42.0
in about:config... what were they thinking?
Allow by IP range? You can probably find a somewhat accurate range for Google and whoever's crawlers.
Google was fortunately that no one sued them for these things before they got big enough to defend themselves. Many tech entrepreneurs haven't been so lucky.
You may ask why a big company like Google isn't doing more to change the CFAA or copyright law. The reason is now that they're big enough, legal grey areas like those in the CFAA (particularly, "what is unauthorized access?", because it's not defined by the statute) can be fully exploited, and Google can sit secure in the knowledge that they'll never be realistically challenged on it; meanwhile, they can then threaten potential competitors for doing the same thing, since a lawsuit against a public corporation takes 10 years and $5MM-$20MM. Anyone who could mount that kind of offense against Google won't, because they benefit from the grey area too; they'll just make some backroom deal with Google and not lose their lucrative, competition-destroying ability to do things that companies with sub-$100M revenues aren't able to do.
From the point of view of the law, that might not matter, but when there's a standardized way to make clear to bots that they're not welcome and you didn't bother to implement it, you'll look pretty silly if you complain.
If someone hasn't exploited a security bug (I mean a real security bug, like a buffer overflow, I don't consider behaviour such as serving up content to certain User-Agents only a genuine "security bug"), and they haven't bruteforced/cracked/acquired a password or private key, and they aren't sending unreasonable amounts of traffic ((D)DOS), it should not be a crime, and the law should be changed to reflect that principle. The law should reflect the common sense of the technically literate, but it doesn't, because it was written by the technically illiterate.
This is the spot we've reached through legislative meddling, and your best bet is either to be a good little consumer and don't make waves or do what you want and don't get caught. Neither of those seem to make a lot of sense in the long run.
So yes, congrats. It's jail for you -- but not for Google. Because they're Google, and you, well, you're not. You'd just better be happy we don't find out about you rooting your cellphone last year. Good grief.
ADD: I know you meant well, and I appreciated the </s> tag, but there was something I didn't like about your comment. Now I know what it is. By making this a big deal, you're increasing the likelihood that this poor schmuck becomes the next "example" some federal prosecutor decides to make. It's not your fault, but it still sucks. Let's hope that doesn't happen.
Edit: I was mis-remembering, the current law is against possession or manufacture of eavesdropping or wiretapping devices, not hacking tools. The EU has been playing with laws against hacking tools, but apparently nothing in the US yet against it.
The law makes it illegal to distribute devices (incl. software) that the design of such [software] renders it primarily useful for the purpose of the surreptitious interception of wire, oral, or electronic communications. Punishable by not more than 5 years and/or not more than $250,000. 18 U.S.C. 2512.
I don't think this blog post qualifies as an "interception" device,... however unauthorized retrieval and recording of another's voice mail messages constitutes an "interception" so who the hell knows. I'm sure you could find a US DA who would argue the falsified User-Agent meant the software is designed to "intercept" communication meant only for Google.
While §202a and §202b punish unauthorized access to or interception of data, §202c extends this threat to obtaining passwords or creating tools in preparation of such an act.
Note that this section has only existed for a few years. I am not aware if white-hat hackers have actually been prosecuted for creating hacker tools.
And by "may", I do mean "may". I don't know. But it's at least possible.
No person shall manufacture, import, offer to the public, provide, or otherwise traffic
in any technology, product, service, device, component, or part thereof, that—
(A) is primarily designed or produced for the purpose of circumventing a technological
measure that effectively controls access to a work protected under this title;
(B) has only limited commercially significant purpose or use other than to circumvent a
technological measure that effectively controls access to a work protected under
this title; or
(C) is marketed by that person or another acting in concert with that person with that
person’s knowledge for use in circumventing a technological measure that effectively
controls access to a work protected under this title.
I mean, obviously it's all quite ridiculous, but also deadly serious at the same time :-(The wording reminds me of a similar section in the German Copyright Law which outlaws the circumvention of "effective" copy-protection schemes.
How an access control scheme can be effective and circumventable at the same time is completely beyond me. :)
Here's how HTTP(S) works: I issue a REQUEST to the web server; the web RESPONDS with it, or denies it. It is up to the web server to respond or deny or do whatever it wants. If the web server is badly implemented or doesn't know what it's doing, it is the webserver's fault.
Remember: it's just a request. I can request 100 dollars from you; the fact that you give them to me does not make me a mugger.
So you go up to a server and lie to it, and it gives you something; is that not acquiring things through deception?
The structure of your argument suggests that e.g. breaking into an ssh server by issuing a login request with a known password which it responds to, isn't illegal. And further, that if data is acquired from the server, there is still no crime - the ssh protocol too is just requests to the server, it's all bits down the line. It's clearly nonsense.
Whether the lock was implemented poorly or you just didn't lock it — doesn't matter.
"If a user agent masquerades as a different user agent, recipients can assume that the user intentionally desires to see responses tailored for that identified user agent, even if they might not work as well for the actual user agent being used"
It's a param exactly equivalent to asking "who would you like to be treated as?" If I say "the pope!" and you respond by kissing my ring, I have not defrauded you.
You're talking about a sign with practically no legal meaning.
This discussion is about a law which does have legal meaning.
To fix your analogy, it would be "If there's a law that says only explicitly authorized cars are allowed in a car wash; else 5 years in prison, and I have a car wash and say 'only red cars' ...". Of course, that still doesn't properly capture this since the intent of the law also matters and that law would be senseless, and so interpretation would be less obvious.
The better analogy is the one below about breaking into homes being illegal, but what if you happen to have a key that fits the lock? (though that also is a bad analogy in its own way).
Basically, making physical analogies for technical matters is rarely correct. It's often the best way to convince non-technical people of a matter without them needing to actually understand it.
If there's anything the tech sector should be able to come to agreement on, it's that lock metaphors in a situation with absolutely no hidden knowledge or private tokens make us all dumber.
The door is locked, the key is right there and all you have to do is pick it up and you can gain access, but that doesn't make you authorised to gain access.
The CFAA and UK CMA may well be overzealous in restricting what should (to technologists) be allowed, but it's what we have as statute in the respective jurisdictions.
The law says "authorized access" and the WSJ authorized Google to access their content in order to index it. The WSJ did not authorize that content to be presented, for free, to you, an end user, necessarily. I don't know which way a court would rule on it, but it's definitely not black-and-white.
Sure, it's technically similar, but the court doesn't care. The court doesn't care if the law makes no technical sense, because it's a law, not a program.
In other words, yes, the WSJ does intend to only give Google access to their content, and not the general public.
But no, the WSJ has not "authorized" Google by anything more official than a bank telling their security guards to let anyone into the vault who is wearing a blue t-shirt.
So yeah, I agree with you, there is a lot of conflation of technical means and the law, but we also shouldn't be granting to the WSJ that they are doing any real "authorizing" here, beyond wishing it and hoping it stays true.
From the ABA: "Exceeds authorized access is defined in the Computer Fraud and Abuse Act (CFAA) to mean "to access a computer with authorization and to use such access to obtain or alter information in the computer that the accesser is not entitled so to obtain or alter."
To prove you have committed this terrible felony, the FBI will now demand that Apple assist in disabling the secure enclave of your device in order to access your browser history. But remember, they only need to do this because they aren't allow to MITM all TLS and "acquire" -- not "collect" -- every HTTP request your machine ever makes. </s>