News for hackers. Stuff that matters.
Re:LinkedIn: It’s illegal to scrape our website without perm... by jalfresi on Monday July 31, 2017 @09:37AM (#14891805)
If you post it on a public network, it is defacto, public.

If you don't want it scraped, take it down, or put it behind a login.

If the user provides the login to a scraper, then the scraper has permission.

Re:LinkedIn: It’s illegal to scrape our website without perm... by dsacco on Monday July 31, 2017 @10:00AM (#14892016)
That's a pretty literalist, one-size fits all approach to policy. I don't think it's a good framework to use for applying ethics considerations.

If I can walk near a pool, should I also be able to run? Is running anything more than faster walking? If I'm allowed to be around the pool walking with my entry ID, should I also be allowed to place my ID on a little motorized car and make it dart around the pool really fast? Should I be able to duplicate my badge, put it on a bunch of little cars and direct them to quickly get all the floaties before anyone else? How about giving them all their own fake IDs? Now all the same questions, except there is a sign that explicitly prohibits all of these examples except for walking.

It seems disingenuous to argue that the automation and rapid increase of a thing should be allowed just because a thing is allowed. That doesn't typically match our intuitive notions of ethics in other parts of society, like driving or walking around a pool. Yes, you can walk around a pool as much as you want, but if you change to running then you have fundamentally altered your behavior through increased capability, not merely done "more of walking" to utilize more of your freedom.

I suppose a natural counterargument to this analogy might be that running around pools is unsafe, and scraping is not unsafe in the same way. But my point here is establishing that a behavior intrinsically changes into a different behavior if you increase the speed at which you're doing it or the capability at which you can do it.

Re:LinkedIn: It’s illegal to scrape our website without perm... by freehunter on Monday July 31, 2017 @10:25AM (#14892252)
The pool has the right to kick you out, same as any website. The pool cannot call the police and charge you with a felony for misusing their resources.
Re:LinkedIn: It’s illegal to scrape our website without perm... by dsacco on Monday July 31, 2017 @10:32AM (#14892317)
Does the pool have any recourse if you proceed to bypass the ban? Do you have to re-enter to pool to bypass it, or does sending in confederates with their own badges to continue your work also bypassing the ban? How about sending in new motorized cars?

The analogy is starting to break down, but I think it's still instructive for the problem of applying a simple first principles approach.

Re:LinkedIn: It’s illegal to scrape our website without perm... by freehunter on Monday July 31, 2017 @12:31PM (#14893446)
There is a legal concept known as "attractive nuisance"[1]. If I have a pool and neighborhood kids come to play and someone gets hurt, it's my fault. Even if I was away from my house and never gave permission (or explicitly forbade them from swimming), if I don't have proper access controls in place, the courts say it is too tempting for the neighbors to just come over and swim. I need to put up a locking gate to keep them out.

Likewise in some high-crime jurisdictions, if you did not lock your car you are liable for it getting stolen or broken into[2]. An unlocked car is too tempting for some people to just walk past and not take it.

I know it might sound crazy but you could make the argument that a massive pool of highly-structured and very valuable data just sitting out in the open is an attractive nuisance and steps should be taken to put it behind a locked gate. Once that requirement has been satisfied, normal trespassing laws apply.

[1] https://en.wikipedia.org/wiki/Attractive_nuisance_doctrine

[2] http://www.cbc.ca/news/canada/montreal/ndg-resident-question...

Re:LinkedIn: It’s illegal to scrape our website without perm... by enraged_camel on Monday July 31, 2017 @05:57PM (#14896238)
Laws like that are ridiculous. You can see that by looking at how the reasoning does not expand to certain areas.

For example, if a woman walks down a dark alley wearing short skirts and gets raped, it isn't her fault. I mean can you imagine if we said "well, she was just an attractive nuisance!" The judge would throw the book at you.

Re:LinkedIn: It’s illegal to scrape our website without perm... by freehunter on Monday July 31, 2017 @06:08PM (#14896318)
They're usually targeted at children who don't know better and don't have the cognitive skills to understand consequences. Usually... but not always.
Re:LinkedIn: It’s illegal to scrape our website without perm... by beobab on Tuesday August 01, 2017 @03:22AM (#14898524)
"An unlocked car is too tempting for some people to just walk past and not take it."

I am saddened by this.

Re:LinkedIn: It’s illegal to scrape our website without perm... by burkaman on Tuesday August 01, 2017 @11:27AM (#14901713)
I don't think it's true, and the source doesn't back it up. Even if it's illegal to leave your car unlocked, that doesn't mean it's your fault if someone steals it. Presumably you'd both be liable.
Re:LinkedIn: It’s illegal to scrape our website without perm... by TheCoelacanth on Monday July 31, 2017 @10:49AM (#14892475)
Bypassing the ban after you have been clearly notified of it is trespass.
Re:LinkedIn: It’s illegal to scrape our website without perm... by jdbernard on Monday July 31, 2017 @11:38AM (#14892954)
That's basically what LinkedIn is arguing here. They have explicitly send hiQ a cease-and-desist letter.
Re:LinkedIn: It’s illegal to scrape our website without perm... by dragonwriter on Monday July 31, 2017 @06:25PM (#14896416)
> The pool cannot call the police and charge you with a felony for misusing their resources.

Once they have withdrawn permission, they can call the police and you can be charged with trespass, though that's usually a misdemeanor rather than a felony.

Re:LinkedIn: It’s illegal to scrape our website without perm... by burkaman on Monday July 31, 2017 @10:32AM (#14892319)
The website can't really kick you out though, it can only kick your agent out and you can trivially create a thousand more. The website can politely ask you to stop just like the pool, but it can't actually do anything if you ignore it.
Re:LinkedIn: It’s illegal to scrape our website without perm... by beobab on Tuesday August 01, 2017 @03:17AM (#14898499)
Except block your IP address, or your user agent, or the pattern your software makes when it connects.

Yes, that will cause potential issues for other people, which is why they tend not to do that, but if you trivially create a thousand more agents, and potentially trigger a degradation of service, how are you different to the people who block junctions at traffic lights?

I'm not keen on inconveniencing people, and "it's not that bad" is a poor argument for doing something that someone has explicitly asked you not to do.

Re:LinkedIn: It’s illegal to scrape our website without perm... by burkaman on Tuesday August 01, 2017 @11:29AM (#14901733)
I think you agree with me. If someone is abusing your website, or even just doing something you've asked them not to do, you can't literally kick them out like you could in person. There's all sorts of techniques for blocking whatever they're doing, but if they're determined and your website is still accessible, they can come back.

That's why it might be reasonable for laws around this sort of thing to be different in the virtual world.

Re:LinkedIn: It’s illegal to scrape our website without perm... by type0 on Monday July 31, 2017 @10:59AM (#14892593)
Sure they can if they know who you are, that means authorization, when L-In will put their pages behind a login they would be able to kick hiQ out.
Re:LinkedIn: It’s illegal to scrape our website without perm... by smokeyj on Monday July 31, 2017 @12:37PM (#14893505)
The way HTTP works is there is a request and response. If LinkedIn does not want to provide a response, they don't have to. If they don't want to accept requests, then maybe the World Wide Web isn't for them.
Re:LinkedIn: It’s illegal to scrape our website without perm... by quickben on Monday July 31, 2017 @05:21PM (#14895995)
So was Napster. Turns out, law>technology.
Re:LinkedIn: It’s illegal to scrape our website without perm... by type0 on Monday July 31, 2017 @10:18AM (#14892181)
> I don't think it's a good framework to use for applying ethics considerations.

What does is it have to do with ethics? CFAA law does not apply here and that is what matters.

Re:LinkedIn: It’s illegal to scrape our website without perm... by dsacco on Monday July 31, 2017 @10:30AM (#14892305)
Ostensibly "ethical considerations" are a precursor to a law being ratified, and will also be used when a law is not sufficiently specified as to obviate the decision. If a party is not denying their activity in court, but rather claiming that it doesn't match existing laws, this would come into play.

But if you contest that or have an issue with the specific terminology I used, perhaps you'd prefer this terminology instead: "I don't think this is a good framework for interpreting the CFAA and establishing legal precedent."

Re:LinkedIn: It’s illegal to scrape our website without perm... by Spivak on Monday July 31, 2017 @10:24AM (#14892233)
I'm interested in this point. I was under the impression that the CFAA would apply here because they've explicitly stated what users are authorized and not authorized to do on their platform. Other than incur shame from the tech community they don't have to actually prevent the unauthorized access or use.
Re:LinkedIn: It’s illegal to scrape our website without perm... by jalfresi on Tuesday August 01, 2017 @04:17AM (#14898719)
The publisher of the information is ultimately responsible for the disclosure of the information. How it is read is of no consequence, as the information has been provided to be read. Certainly, there are issues about resource usage i.e. heavy readers, such as scrapers, but throttling is a perfectly acceptable approach to overuse of resources for both parties, as access is still available over managed resources.

The nature of the original complaint is authorised access to publicly published information. Again, if you do not want people to read publicly published information, do not publish it publicly.

And as a side note - we don't need inappropriate analogies; the web is real, we can discuss the real issue.

Re:LinkedIn: It’s illegal to scrape our website without perm... by aeorgnoieang on Tuesday August 01, 2017 @09:11AM (#14900246)
What LinkedIn is doing seems more like a pool threatening to sue people that look at the facility itself, from outside of their property. It's not clear whether viewing a public website is more (reasonably) similar to entering someone's property versus looking at their property from outside of it. To me, it seems much more like looking at it from outside. But, fortunately for LinkedIn, they can prevent people from viewing their site! (They just have to figure out for whom they want to do so.)
Re:LinkedIn: It’s illegal to scrape our website without perm... by rjp0008 on Monday July 31, 2017 @10:20AM (#14892211)
I like this analogy. I think maybe the answer would be to include technical limitations, like API limits so scrapers can only work as fast as a human. Or maybe captcha interaction like select all images of a car could be integrated? I think it should be up to the pool operator to make sure people follow the rules they put in place, not call the cops because someone is running.
Re:LinkedIn: It’s illegal to scrape our website without perm... by williamle8300 on Monday July 31, 2017 @10:44AM (#14892429)
Yes! Exactly. This is the best, most clear argument for how cases like this should be handled.
Re:LinkedIn: It’s illegal to scrape our website without perm... by sgt101 on Monday July 31, 2017 @10:05AM (#14892069)
What if the user is provided with permission to view medical records - your medical records for example? Would it be ok to scrape and sell those? What about children's school profiles and records?
Re:LinkedIn: It’s illegal to scrape our website without perm... by wolco on Monday July 31, 2017 @12:11PM (#14893197)
Selling medical records would fall under a different law same goes for sharing credit #s cards you collect.

I think the issue is around public data here. If you made medical records available to the public you would be in trouble in many countries. Sharing those may land you in the same trouble as the website who initially posted it under that law. The person shouldn't be charged with unauthorized access because the materials are public.

Re:LinkedIn: It’s illegal to scrape our website without perm... by jalfresi on Monday July 31, 2017 @10:55AM (#14892544)
See the first line.

If there is a problem with that, see the second line

Re:LinkedIn: It’s illegal to scrape our website without perm... by sgt101 on Monday July 31, 2017 @01:32PM (#14894142)
For shame!
Re:LinkedIn: It’s illegal to scrape our website without perm... by wastedhours on Monday July 31, 2017 @10:02AM (#14892039)
At the risk of bringing out the "you wouldn't download a handbag" brigade - you could make the argument that just because I leave my bike unlocked on the street doesn't give you a right to ride it off.

Even if you go for "forgiveness rather than permission", if you ride it a second time after I've told you I don't want you riding it, then you're in the wrong.

Obviously, there are philosophical arguments to be made about the status of information online, but if the information were provided to LinkedIn, to be used on LinkedIn properties, I as a user would take umbrage with it being taken by a 3rd party, even if it were viewable publicly.

Re:LinkedIn: It’s illegal to scrape our website without perm... by aphexbr on Monday July 31, 2017 @10:14AM (#14892140)
The bike example is a very poor analogy - the data isn't removed from LinkedIn, it's merely copied. It doesn't matter how many times the site is scraped, the data is unchanged and still available. A better analogy would be me taking a photo of the bike while walking past. It shouldn't matter how many times you tell me to stop, if it's on public property you can't really stop me from taking the photo.

If you have an issue with that, you should be moving your bike somewhere people can't take a photo from the public street. Not have someone creatively interpret a law that says where I am is suddenly not public property, because you asked me to stop using my camera.

Re:LinkedIn: It’s illegal to scrape our website without perm... by type0 on Monday July 31, 2017 @10:30AM (#14892307)
Not sure why you're being down-voted, the Web were designed to be public. If you want to prevent me from taking pictures of the exterior to your Cafeteria on the public street you'd have to build a wall / put it behind a login. But then don't complain that you are loosing customers because they can't see it or can not find information to your site through search engines.
Re:LinkedIn: It’s illegal to scrape our website without perm... by wastedhours on Monday July 31, 2017 @10:27AM (#14892266)
Ok, if I left the prototype for a new bike on the pavement, and you came along and 3D printed an exact replica. Sure, the original still exists, but you just violated trademark/registered design laws because it was there. It's not the same as a photo because a photo of a bike doesn't give you the same value as the actual bike, whereas scraping the content of a webpage does give you the same value.

Perhaps my analogy wasn't great, but the grey area is around going to LinkedIn's server (whether or not this is "public" or their property they allow you access to is another philosophical question, though in the eyes of the law it appears it's the latter), deliberately extracting value from it, and then getting annoyed when you're asked not to.

Inherently it seems as though it's the old question of whether a server is public, or private but accessible (like those POPs [0] there was a thread on recently).

[0] https://en.wikipedia.org/wiki/Privately_owned_public_space

Re:LinkedIn: It’s illegal to scrape our website without perm... by Chris2048 on Monday July 31, 2017 @12:39PM (#14893522)
> you just violated trademark/registered design laws

If it has those. The data on LI (other people's employment histories) is not its own IP.

Re:LinkedIn: It’s illegal to scrape our website without perm... by hashkb on Monday July 31, 2017 @10:19AM (#14892194)
What if I paint the bike artfully, put a copyright notice on it, and you sell the photos you took?
Re:LinkedIn: It’s illegal to scrape our website without perm... by Spivak on Monday July 31, 2017 @10:26AM (#14892263)
Standard IANAL.

That would be copyright infringement because photographing a copyrighted work is considered reproducing the work under the law.

Re:LinkedIn: It’s illegal to scrape our website without perm... by rootlocus on Monday July 31, 2017 @10:41AM (#14892393)
> A better analogy would be me taking a photo of the bike while walking past.

Taking a photo of the bike is also a poor analogy. Scrapers don't take one photo, they take photos of all the bikes. And scrapers don't keep the photos for themselves, they sell them for a profit. Also, the original bike isn't parked, it's placed in a gallery (probably with an admission fee? I don't know the business model of linkedin).

Re:LinkedIn: It’s illegal to scrape our website without perm... by aphexbr on Monday July 31, 2017 @10:55AM (#14892541)
The number of photos is irrelevant to the analogy, though, as is what people do with the photos afterwards. If the bikes are visible from the public street, people can take as many pictures of every bike they want, and then make money from them if they want. It doesn't affect the owners' usage of the bike (unlike the original analogy, where the owner loses access, which was what I was trying to correct)

Physical analogies for this kind of thing are always flawed, it's just dishonest/misleading to pretend that copying data is ever analogous to taking a physical object (the owner of the original is never deprived of the original when data is copied).

"I don't know the business model of linkedin"

Most of it is selling premium features to recruiters and other businesses. I'm not sure if Hi-Q's service interferes with that or not, but LinkedIn should not be trying to have their cake and eat it by leaving things in public then complaining when the public accesses it in a way they don't like.

Re:LinkedIn: It’s illegal to scrape our website without perm... by rootlocus on Monday July 31, 2017 @11:10AM (#14892709)
> The number of photos is irrelevant to the analogy

Actually, the size of the data and the number of requests is very relevant. More data means more information, means more money. It also means more bandwidth and processing power required to process requests. You're not taking a photo of the bike, you're asking the bike to give you a photo of it.

> it's just dishonest/misleading to pretend that copying data is ever analogous to taking a physical object (the owner of the original is never deprived of the original when data is copied).

Leaving LinkedIn aside, possession of the original data is never the issue with digital piracy. It's a straw man. The hurt occurs when people benefit from the work the original author put into creating that data without proper compensation. Just because you can clone my gizmo (which I spent years working on) without taking the original one doesn't mean you're not hurting me. That gizmo could give me an advantage you wouldn't otherwise have. I place hours of working into something that doesn't put food on the table because you can clone my work, but I can't clone my food.

There's a reason an empty CD costs 50c but a music album costs $10. You're not paying for the physical medium. You're paying for the IP. And yes, digital distributions are cheaper because of this, but that doesn't make them free.

> Most of it is selling premium features to recruiters and other businesses.

I'd say it's pretty obviously interfering with their business model.

> LinkedIn should not be trying to have their cake and eat it by leaving things in public then complaining when the public accesses it in a way they don't like.

LinkedIn could ban IPs that make unreasonable number of requests in a short amount of time.

Re:LinkedIn: It’s illegal to scrape our website without perm... by aphexbr on Tuesday August 01, 2017 @08:59AM (#14900160)
If LinkedIn are being that negatively affected by a single scraper, they should deal with it - block it, only allow a specific number of requests from an IP per day, anything that doesn't involve lawsuits. The problem is them trying to pretend that publicly visible content is really private if they say so, without them trying to protect it in any real way.

"The hurt occurs when people benefit from the work the original author put into creating that data without proper compensation"

Not necessarily. If I'm paying for print of some imaginative artwork that was created using the picture of the bike, that doesn't mean the bike owner lost anything, even if he spent time building the bike with his own hands. Similarly, if the only reason why people paid Hi-Q was for the extra work that they put in, LinkedIn didn't lose money because people would not have bought their product without that extra work.

There is certainly an argument that Hi-Q should have licenced the content first, but it's public data. If they want to make licence deals, don't put it in the view of the public street then whine when people are documenting what's in public.

"It's a straw man."

No, the straw man is pretending that a copy is the same as theft. Theft is theft because someone is depriving you of the original, not because you imagine you might have had more sales if the copy didn't exist. There's a reason why there are different words for different things, and pretending that a copy is the same as taking a physical object it a lie. Period.

"I place hours of working into something that doesn't put food on the table because you can clone my work, but I can't clone my food."

But, you put the price up too high, so I opted not to buy it. Maybe borrow the CD from a friend, or listen to something else. Or, you decided I couldn't buy it in the format or region I wanted. There are real issues, but pretending that a copy = a lost sale is utter bull that's been debunked time and time again, yet is regularly repeated by people trying to inject emotional arguments instead of facts.

"I'd say it's pretty obviously interfering with their business model"

Then perhaps they should address the business model or not put their content out there in public unprotected if it's that valuable to their income.

"LinkedIn could ban IPs that make unreasonable number of requests in a short amount of time."

Yes they could. Which would not have to involve the courts in any way. Or, they could protect the content in some other way that (for example) requires a log in and adherence to T&Cs, with which they could easily kick violators off their site for non-compliance.

The issue is that LinkedIn are trying to have it both ways - gathering the benefits of public content while blocking others who use the now-public content in ways that are usually acceptable for public content to be used. Sorry, not acceptable, you pick one - take the content away from the public street or accept that some people will use what has been shown to the public.

Re:LinkedIn: It’s illegal to scrape our website without perm... by rootlocus on Tuesday August 01, 2017 @10:34AM (#14901017)
> If LinkedIn are being that negatively affected by a single scraper, they should deal with it - block it, only allow a specific number of requests from an IP per day, anything that doesn't involve lawsuits. The problem is them trying to pretend that publicly visible content is really private if they say so, without them trying to protect it in any real way.

With this, I agree 100%.

> No, the straw man is pretending that a copy is the same as theft. Theft is theft because someone is depriving you of the original, not because you imagine you might have had more sales if the copy didn't exist. There's a reason why there are different words for different things, and pretending that a copy is the same as taking a physical object it a lie. Period.

That's just pedantry. The debate isn't between "copy" and "theft", it's between "theft" and "copyright infringement".

> But, you put the price up too high, so I opted not to buy it. Maybe borrow the CD from a friend, or listen to something else. Or, you decided I couldn't buy it in the format or region I wanted. There are real issues, but pretending that a copy = a lost sale is utter bull that's been debunked time and time again, yet is regularly repeated by people trying to inject emotional arguments instead of facts.

This is wrong on so many levels, I'm not sure there's any point in continuing this debate. Are you accusing me of using emotional blackmail instead of facts because I point out that "you can clone my work, but I can't clone my food"?

I'm not using myself as an example because I want pity. I'm doing it because it's easier in writing, and because I'm a software developer.

My work takes hours of hours of time and effort (not accounting the hours I spent in school). If it' ok for everyone to clone my work, I won't make any money from it. We still live in a society where goods and services are exchanged with money. I exchanged my hours of work for no money, but I can't exchange no money for basic living necessities such as food. There's no feelings involved here. In the current economy, work going in, and no food coming out is not a viable business model. And if nobody payed for digital content, there would be a lot less digital content.

> pretending that a copy = a lost sale is utter bull that's been debunked time and time again

This is another straw man. Whether or not an illegal copy is or isn't a lost sale is irrelevant. You don't have the right to make that copy in the first place. If everyone made illegal copies, there would be no sales. So then why should only some be entitled to illegal copies? There isn't a distinction between people who can make copies and people who must pay for copies, so either everyone must pay for copies or no one must pay for copies. That's how law and economy work. You can't make exceptions by yourself. Either everyone is allowed, or no one is allowed. And for digital content that is for sale, no one is allowed illegal copies. If laws are made that allow poor people to receive goods for free, these laws must address both digital and physical goods.

Re:LinkedIn: It’s illegal to scrape our website without perm... by jalfresi on Monday July 31, 2017 @10:54AM (#14892533)
So? How does this impact the bike owner in anyway?
Re:LinkedIn: It’s illegal to scrape our website without perm... by rootlocus on Monday July 31, 2017 @10:56AM (#14892554)
The owner makes money from the admission fee clients pay to see the bike. Taking photos and selling albums defeats that purpose.
Re:LinkedIn: It’s illegal to scrape our website without perm... by mywittyname on Monday July 31, 2017 @11:44AM (#14893006)
But the owner is offering the images up for free to the public. hiQ is taking those publicly-available photos and annotating them with, "red bike", "pink bike", "broken bike", "professional bike", etc.

It's clearly a value-add and not theft.

Re:LinkedIn: It’s illegal to scrape our website without perm... by quickben on Monday July 31, 2017 @05:28PM (#14896034)
It depends.

You have to keep in mind that an entire generation was brainwashed that personal data isn't that "personal", so Google, FB and the rest can have amazing profits.

Most of these discussions are stained by general unawareness of the privacy and copyright law.

Ofc because the value of the data supplier (usually a single person, etc) these never reaches the courts, which just reinforces the ongoing misconceptions.

If you really want to test this, try copying the content from Google, Facebook hiQ or whoever that's big enough to go after you.

But people somehow believe that it's okay for businesses to do what regular persons aren't allowed to.