https://en.wikibooks.org/wiki/UK_Database_Law#Database_Right
If you scrape, and effectively reconstitute a database, then so long as the database originally had a "substantial investment" in it's "obtaining, verifying or presenting the contents" then yup... you have breached the database right, which is a modified form of copyright.
You may access said database (via the web), but as soon as you start reconstituting the database from scraping... you're in breach.
It's a law, it is illegal in the UK, I'm sure most countries have some equivalent law on their books, all of the EU does. The law looks recent, but UK copyright and patent used to cover it, the 1997 date is just a separate statute to clarify the position.
The World Copyright Treaty of the WIPO, which the US also signed, enforces in Article 5 that every member country has to have a database law of this kind.
________________________
> Compilations of data or other material, in any form, which by reason of the selection or arrangement of their contents constitute intellectual creations, are protected as such. This protection does not extend to the data or the material itself and is without prejudice to any copyright subsisting in the data or material contained in the compilation.
http://www.wipo.int/wipolex/en/treaties/text.jsp?file_id=295...
> United States of America
> Signature: April 12, 1997
> Ratification: September 14, 1999
> In Force: March 6, 2002
________________________
If Feist v. Rural occurred now and Rural, like most companies, kept their information in a database online, Feist would lose not for copyright infringement, but for exceeding authorized access to Rural's server.
This isn't even true metaphorically. It's like a shop front: there may be public access, but it is NOT public property.
Taking the store metaphor further, it would be more like you knocking on the front door of a clothing store and the store owners open the door and throw every possible piece of clothing at you, shirts, shorts, underwear, including coupons to "partner" stores, when all you wanted was a pair of pants.
Upon knocking, if the store owner hands you instructions on how to enter their store and interact with their products in a personalized shopping experience, that would be one thing. But when the clothing owner throws everything at you at once, what they flung at you is for all practical purposes public property.
This is called "clickwrap". There is usually a notice in the footer of each page that says something like "By using this site, you agree to our Terms of Service." Typically, this kind of notice has been held enforceable. More recently, judges have been demanding that such notices be placed more prominently before they're held enforceable (e.g., somewhere above the fold), but that's it.
>Imagine a hotel that makes guests sign a document saying they will not make photographs of the building. If I'm not a guest, I can take photographs of it and I can't even know that would be illegal.
The reasonable laws that exist in meatspace are not applicable online, because once you hit someone else's server, you're considered to be on their property and they have the right to control what you do there. There is no "public property" from which to safely stand and take photographs in the internet.
Also, photographs of structures may not be free to use. Architectural copyrights went into effect in the early 90s and have a term of either 90 or 120 years. Thus, if you take a photograph of a building built in 1991 and the year is not yet 2111, there is a chance that the architect can claim infringement.
1. Total privacy, they will not track me activity on their website, including any logs.
2. They will send me a cashier's check for $1,000 for each byte that they send to me.
3. They will provide me with Mana Sakura's cell phone number.
I'm still waiting for checks and a phone number.
It is ridiculous. Something like "pagewrap" can't trump the consumer protections that apply to a physical good like a book, it would be laughed off. But the law doesn't contemplate network access so reasonably.
The architect can claim infringement all they want, they don't have a case. From https://www.law.cornell.edu/uscode/text/17/120 :
The copyright in an architectural work that has been constructed does not include the right to prevent the making, distributing, or public display of pictures, paintings, photographs, or other pictorial representations of the work, if the building in which the work is embodied is located in or ordinarily visible from a public place.
This is an important caveat to architectural copyright, however, so thanks for clarifying.
See also https://mentalhealthcop.wordpress.com/2013/09/20/place-to-wh... which shows the same ambiguity exists in the UK.
IANAL but this seems perverse. In no meaningful sense am I on corporate property when my computer in my house sends signals to another computer, formatted so that they will be re-sent in turn to a series of other computers, the last of which decides on its own based entirely on the signal it receives from the penultimate host to send a "response" to a different series of other computers, the last of which is my computer in my house.
Surely there are better ways to enforce IP restrictions than this tortured analogy of networked computing to physical location?
Even if we entertain a distinction between browsewrap and clickwrap, browsewrap is generally enforceable, especially after minor modifications to placement and/or font size.
Even for sighted people, the notice is often easy to miss - and this is by design.
I don't think many websites have a secret ToS that they hope you won't read, I think most of them don't even know what their own ToS say. I signed my lease on a site with an explicit checkmark for ToS that said I agreed I would only use exactly IE7 to use their site.
I suppose I can do no better than quote from the Wikipedia page I linked:
> The Second Circuit then noted that an essential ingredient to contract formation is the mutual manifestation of assent. The court found that "a consumer's clicking on a download button does not communicate assent to contractual terms if the offer did not make clear to the consumer that clicking on the download button would signify assent to those terms."
The same page cites a number of cases where a "browsewrap" agreement was found unenforceable and only one where one was found enforceable - and the latter, for what it's worth, involved a sale taking place through the website rather than anything resembling passive browsing. Of course there exist other cases not listed; and there are situations that muddle the distinction between clickwrap and browsewrap. But at least, the very common pattern of, as you said, burying "a notice in the footer of each page" without anything vaguely resembling active consent, as practiced by probably the majority of commercial websites on the internet, seems to pretty clearly fall on the unenforceable side of the line based on those precedents.
https://en.wikipedia.org/wiki/EBay_v._Bidder%27s_Edge
The courts have generally disagreed with that interpretation.
No, its not. It may be in public view, but that's a different issue.
This is a gross misunderstanding of how the internet works.
So I'm not so sure that police will escort you out of a Walmart because they caught you taking a picture of the parking lot with your smartphone.
If you're entering a country, do its laws not apply to you until you've seen a copy of them? "Oh, sorry, no one told me theft is illegal here. Where does it say that? Oh, I see. Okay. I'll stop now. Thanks for letting me know."
If you cross the border without necessary documents, does that country have no right to detain you, simply because you haven't checked the laws?
Just because a website is visible and public doesn't mean its content is public domain. It just means that your first order of business as a user should be to check the terms of service. Sure, most people using a website probably don't need to--same as not needing to check a country's stance on murder--and so can just use the website as intended without violating the terms. But when you plan on using it in a way that might not be intended, and you don't check the terms of service, well, that's on you.
Also we don't even know how many laws there are in the United States for I'd say knowing the content is impossible.
I don't need to check your terms of services if I'm doing something that I'm allowed to do by law anyway; the TOS cannot deny me those rights (they might, of course, grant me additional rights provided that I follow certain conditions).
The entire point of protocols is to precisely define the terms of communication. The status code is '200 OK', not '200 OK/Asterisk'. But of course if lawlers didn't force themselves into the situation, they'd be out of jobs.
As an aside, I'd really like to see a browser plugin that would scrape sites in the normal course of access, storing the proceeds in a distributed public database.
This would be copyright infringement, since the content of the page is a substantive unique work that is automatically copyrighted by its author. A site that doesn't want you scraping its content is not going to want you posting dumps of its pages. Much like BitTorrent, they'd get into the protocol and send subpoenas to the ISPs behind the IPs that serve their pages, and use that info to sue the customer.
When my company was shut down by a legal threat related to scraping, I did suggest to my lawyer that we create something like a browser extension that would grab the data we needed out of normal client-side browsing sessions. This wouldn't be as nice as controlling the flow of information ourselves but it would've worked OK. My lawyer strongly suggested avoiding that as it could've been construed as conspiratorial conduct that would've made criminal prosecution more likely.
Perhaps this illustrates the fungibility of the legal system: it's an inherently human construct that pits a plaintiff against a defendant, and given a big enough warchest and persuasive-enough arguments, catastrophe can be avoided -- by Google; perhaps not by you, me, or someone else.
The main difference when Google was small was that Google was not dependent on any data source in particular, so even if someone denied their robot or sued them, they could cease and desist without affecting the overall value of their offering. This is different if you are getting data that is only available from one or two sources.
Now, the main difference is that Google is one of the biggest companies in the world, and they'll sick an army of $1,000/hr lawyers on you if you even think about taking legal action against them. The only people who can afford to fight are other big companies, but that's not going to happen because they all depend on breaking the CFAA for their own purposes and then using their position as a huge company to bully small innovators.
https://en.wikipedia.org/wiki/Field_v._Google,_Inc.
There are similar rulings for thumbnail images:
https://en.wikipedia.org/wiki/Perfect_10,_Inc._v._Amazon.com....
And of course books:
https://en.wikipedia.org/wiki/Authors_Guild,_Inc._v._Google,....
Google's primary out here is its reputation (not guarantee) for obeying robots.txt. If Google indexed a page that disallowed it in robots.txt, the case would be much stronger. There's also the unofficial out, which is that judges think Google is a cool large company, so they rule in their favor based on their personal biases.
Fair use is a case-by-case basis, so you can't say that Google's infringing conduct is generally accepted to be fair use. The EFF had to take on Universal in Lenz v. Universal Music Group, and that went up to the Supreme Court. That's how individuals are left to assert their fair use rights.
>Fair use is a case-by-case basis, so you can't say that Google's infringing conduct is generally accepted to be fair use.
There is so much wrong with this statement. For one, how can you call something infringing at the same time you point out that nothing has been proven? That simply defies all common logic.
Secondly, in general terms, the activities in question have been found to be non-infringing by the courts. Sure fair-use is case-by-case but if you're operating within similar parameters as a previously litigated case, then the legal risk is immensely reduced.
I don't disagree with your assertion that the legal system greatly favours the well monied/connected (I don't think anyone would). But you can't claim it to be fact that Google Search is infringing anything with little to no evidence or rulings to cite. Unless you're just stating an opinion in which case you should clearly indicate that.
Fair use is an affirmative defense. Google admits that it copies content without legal license to do so, but claims that said copies are non-infringing under fair use exemptions. I guess you're probably correct that it's no longer appropriate to refer to Google's behavior specifically as "infringing", just "copying without authorization", which, for those of us without $5 million to commit to a legal team, means "infringing". I will try to remember the special standard of law which has been allowed to Google and refer to their copying only as "unauthorized" and not "infringing" in the future.
If you review the points summarized in the Wikipedia articles you helpfully linked, you'll see that Google's defense is mostly "Yeah, but we're Google".
In Field, "the court found that the plaintiff had granted Google an implied, nonexclusive license to display the work because of Field’s failure in using meta tags to prevent his site from being cached by Google.", i.e., because Field already knew Google existed and knew there was a standard way to prevent its access but chose not to employ it, he gave Google an implied license.
Who else does that work for? Can I send an email to Netflix and tell them "Hey, if you don't want me to copy your shows, please add this in your page's HEAD element: <meta name='please-dont-download-my-shows-sir'>"? No?
I understand there are other criteria which were used to decide if Google's use was specifically infringing in addition to the implied license. Just demonstrating that Google is getting favored treatment from the judiciary that would not be available to a normal entity.
In Perfect 10 [0], the judge even explicitly indicated that he was loathe to find Google's use of thumbnails infringing because he didn't want to "impede the advance of internet technology", but that he felt the law obligated him to do so (his ruling in that matter was overturned on appeal, when the Ninth Circuit found Google's usage non-infringing). What if the defendant had been some company perceived as less technically advanced than Google? This is probably as close as you can get to an explicit statement of favoritism. The Ninth Circuit also rejected Perfect 10's claim that RAM copies were infringing (which was not the case with an unlucky non-Google company discussed further down).
What if I started indexing and rehosting thumbnails? I can assure you that I would get C&D'd almost immediately and I would be forced to shut down because I can't afford to pay lawyers for 3 years while the case works through the system (and to be honest, I'm surprised it only took 3 years). And even if I could, with a reputation less sterling than Google's, there's no reason to believe that a judge would rule in the favor of one useless guy instead of a big company. A judge would look at the case and say "Google's use was fair because it provided a public service [actually cited as part of the justification in most of your linked cases], but this guy is just using it for a few hundred people, it's definitely unfair, he owes that company more money than he'll make in his life, case dismissed".
There are many such cases on the books. I don't know if Google has a direct connection to the reptilian overlords or what, but it seems in most cases where they're not involved, the good side loses.
In Craigslist v. 3Taps, while primarily a CFAA case, 3Taps was found to be infringing copyrights by sampling Craigslist postings in order to allow its clients to plot them on a map. Being a "public service" or a "referential use" didn't matter for them. They were raked over the coals, and it's been that way with most cases.
In Ticketmaster v. RMG Technologies [1], RMG was found to infringe just by parsing a page. "Defendant's direct liability for copyright infringement is based on the automatically-created copies of ticketmaster.com webpages that are stored on Defendant's computer each time Defendant accesses ticketmaster.com. [...] Defendant contends [...] that such copies could not give rise to copyright liability because their creation constitutes fair use[.] [...] Defendant's fair use defense fails."
The case specifically discusses how, despite the precedent in Perfect 10, since the Defendant is not Google, it is bound by a site's Terms of Use and copyright law, and RAM copies, which are specifically non-infringing for Google, were infringing for RMG.
Very similar findings were made in Facebook v. Power Ventures, and the founder was left holding a bag of $3 million in personal liability.
This is a thread about the legality of HN users scraping. It seems Google is the only entity capable of making unauthorized copies and then getting courts to agree that it's fair use. For the rest of us, it's infringement, which carries stiff penalties (and this doesn't even broach the CFAA portion of the issue).
So when I say "infringing", I mean something that would be considered infringing if you aren't Google. It's apparently only infringement if the judges involved don't personally use your site and don't have to worry about personally suffering the consequences of not having access to it. :)
[0] https://www.eff.org/document/perfect-10-v-google-ninth-circu...
[1] https://scholar.google.com/scholar_case?case=147697505884223...
>Can I send an email to Netflix and tell them "Hey, if you don't want me to copy your shows, please add this in your page's HEAD element: <meta name='please-dont-download-my-shows-sir'>"?
Actually, under fair use you certainly can make a personal copy (see Betamax case). If you distribute the work you would likely run afoul of the criteria summarized above.
The robots.txt relevancy is being over stated in your argument. The main criteria used in this case is summarized above. The fact that Google provides an opt-out mechanism is a secondary, supporting argument.
>What if I started indexing and rehosting thumbnails? I can assure you that I would get C&D'd almost immediately
A determination of infringement would depend entirely on the context as related to the afore mentioned criteria. The fact that someone might try to sue is a product of the terrible system in general and you're absolutely right - as with any legal matter the entity with the deeper pockets can often bully the other guy into submission.
>In Craigslist v. 3Taps, while primarily a CFAA case, 3Taps was found to be infringing copyrights
My understanding is that the copyright part of the case was thrown out [1] and thus was settled solely around CFAA matters.
>In Ticketmaster v. RMG Technologies , RMG was found to infringe just by parsing a page.
I agree that the logic used for the judgement is absurd (for reasons that are plainly obvious to any HN user). But it's less clear whether the case would meet fair use criteria outlined above should it have come to that. My guess is that it wouldn't qualify since the usage affects the copyright holders ability to make money on the work and doesn't meet any of the other criteria for Fair Use.
>Facebook v. Power Ventures
This is not a case involving a defense of fair use (as far as I can tell). Facebook even acknowledged the users owned the data and had a right to it. The defendant was actually found to be violating CFAA and CAN-SPAM acts.
>It seems Google is the only entity capable of making unauthorized copies and then getting courts to agree that it's fair use. For the rest of us, it's infringement
Provably false [2]. It sounds like perhaps your personal experience has soured your opinion on the matter? That's understandable. But none of the evidence you've cited supports the argument that Google is infringing copyrights in its core activities nor that Google is the only entity where copyright laws and fair use legislation don't apply.
PS: To be clear, my argument revolves specifically around copyright infringement and fair use. I don't have enough understanding of other, separate legislation like CFAA to comment on that except to say that it seems overly broad and unrealistic. But that's another topic. I'm specifically arguing against calling Google a copyright infringer in a broad sense which is what you've done. That's not been proven.
[0] https://en.wikipedia.org/wiki/Fair_use#U.S._fair_use_factors [1] https://techcrunch.com/2013/04/30/craigslist-3taps-lawsuit-d... [2] http://fairuse.stanford.edu/overview/fair-use/cases/
Yes, I understand that the criteria for fair use is defined in the statute. What I'm saying is that like most things brought before judges, arguments can be made either way, and judges seemingly favor Google but not smaller defendants. Thus, while the RAM copies of web pages made by Google are fair use, those made by RMG aren't.
If you look at the Ninth Circuit's ruling in Perfect 10, the length they stretch to reverse the District Court's finding of thumbnails as infringing is ridiculous. It's pretty clear that thumbnails are direct infringements and that you don't invalidate the copyright or create a truly "transformative use" by making it smaller and adding it to an index. Perfect 10 was certainly of this opinion, and I'm sure they saw a real impact to their revenue.
Over the years I've learned that no position is too high to disregard the human factor. 99% of the time people are going to act primarily to their own benefit and work backwards to find rational (or rational-sounding) arguments to justify it. Judges are politicians and they're very image-conscious. None of them wants to be the one to make Google Image Search useless.
You seem to be saying that since Google's use was found non-infringing in these cases, its use is objectively non-infringing. I don't agree with this. Rather, I think that Google's conduct is a pretty plain violation of the relevant statute(s) and that most of it is not covered under fair use, the way the laws are currently written. I think that judges apply the statute in full force when smaller defendants present, but that they have a bias for Google (which is really a bias for themselves, since they know that serious backlash awaits the judge who puts the kabosh on it) that causes them to contort the law pretty heavily so that they can rule the way they want to.
>Actually, under fair use you certainly can make a personal copy (see Betamax case).
See, we were on the right track before we got into networks. Since then, the rulings have been pretty darn bad. The modern "Betamax case" may well have been American Broadcasting Cos. v. Aereo, Inc. [0], and it wasn't a win for us.
Note also that separate from the copyright concern, the DMCA makes it illegal to circumvent a copy protection device (or indeed, even to teach another how to do so). Since Netflix employs DRM, even if there is a fair-use right to a copy of a Netflix program (which is by no means certain), you'd probably have to break the DMCA to obtain it.
>The robots.txt relevancy is being over stated in your argument. The main criteria used in this case is summarized above. The fact that Google provides an opt-out mechanism is a secondary, supporting argument.
I disagree. Google has been able to discharge all CFAA claims because the judges have said "Well, you knew there was a way to stop it." If that's the logic, I'll happily inform the parties I may scrape that there's a way to stop it.
>A determination of infringement would depend entirely on the context as related to the afore mentioned criteria.
Yes, I understand that the judge would write a report that appeared to consider the relevant criteria. The real question is, would that judge be willing to make the same logical contortions that other judges have made for Google?
I think that he would just go in favor of his biases, and right now we have a judiciary that is heavily biased against the little guy from the start, and this is only exacerbated by an inability to retain hotshot lawyers.
>My understanding is that the copyright part of the case was thrown out and thus was settled solely around CFAA matters.
The only portion of the copyright claim that was dismissed was Craigslist's claim that it owned an exclusive license in the scraped content. This was based on a short-lived ToU update that was specifically intended to strengthen Cragislist's case in this instance. The remaining copyright-related claims were allowed to stand, including a claim that Padmapper had violated a copyright Craigslist holds on the collection of advertisements (rather than on the advertisements themselves). [1]
>[re: RMG] I agree that the logic used for the judgement is absurd (for reasons that are plainly obvious to any HN user).
If you agree the logic was absurd, you agree that a copy of the page that exists in RAM for microseconds does not qualify as a protected copy any more than the reflection of an image on one's retina qualifies. As a "copy" that should be ineligible for copy protection, it doesn't matter if it qualifies for fair use (and I don't necessarily agree that it wouldn't).
> [re: Facebook v. Power] This is not a case involving a defense of fair use (as far as I can tell).
Correct. I was including it because it's an example of Google getting another free pass for stuff that shuts others down, which is the CFAA. CFAA claims are raised against Google in at least Field and Perfect 10, and they get dismissed based on the judge's assumption that the plaintiff knows about the special steps Google makes you take to stop them from violating the CFAA, the absurdity of which we've already discussed.
My wording that the "findings were very similar" was definitely bad since a different law was in play. I meant they were very similar in nature, not in fact. That said, it's likely the only reason that the cached pages weren't considered infringement is that Facebook didn't bring it up.
>But none of the evidence you've cited supports the argument that Google is infringing copyrights in its core activities nor that Google is the only entity where copyright laws and fair use legislation don't apply.
Again, I'm discussing this from a practical position, not one that is strictly compliant with legal theory, where judges always enforce the law with perfect equity, and in which anything a judge (or jury) finds becomes Official Truth de-facto.
From a textbook perspective, sure, everyone has all the same rights and the legal system is always applied equitably. I simply don't believe that has borne out in practice when it comes to internet-centric companies that aren't household names.
It seems that the things Google does are considered infringement when other people do them. Thus, it behooves to know the actual law and follow it, even if Google gets a free pass, since we can't rely the judiciary to interpret the law favorably for us.
RMG is a great example because it occurred after Perfect 10, and the same argument against RAM copies was raised in both cases. It's apparently fair use if Google scrapes your page to download and rehost all of your images, but it's not fair use to read out non-copyrightable factual data unobtainable from any other source (like ticket prices and event times) and rehost it nowhere. Sure.
The alternate lesson here is to focus on getting really big and powerful really quickly, and making sure you cultivate a good public image, so that judges are afraid to rule against you in ways that would affect a product offering upon which millions of people depend. That seems to have worked for most big internet companies, actually. Definitely worked for Facebook and Google.
[0] https://en.wikipedia.org/wiki/American_Broadcasting_Cos._v._....
[1] http://www.dmlp.org/sites/dmlp.org/files/2013-04-30-Order%20... pgs. 9-16
They don't even need to do that. They just cheerfully agree to not scrape you, and wait for you to come back and beg to be re-instated when your search traffic plummets.
I think imbuing technical protocols with legal implications would be even worse than the current situation since then changing anything on a protocol would require changing the law and getting a protocol implementation slightly wrong would carry real-world legal repercussions on the order of licensing your work in the public domain rather than retaining copyright. Let the lawyers make the law and check the human terms of service before using the data. Trying to out-lawyer the lawyers is like challenging a hedgehog to a butt-kicking brawl.
"The User-Agent request-header field contains information about the user agent originating the request. This is for [...] the tracing of protocol violations [...]. User agents SHOULD include this field with requests"
Many scrapers disregard this part of the protocol. Of course, whether a headless browser should send a different UA is an interesting question.
Imagine a hotel that makes guests sign a document saying they will not make photographs of the building. If I'm not a guest, I can take photographs of it and I can't even know that would be illegal.