News for hackers. Stuff that matters.
Re:Web scraping case fails under Supreme Court's Dastar doct... by alkonaut on Monday November 19, 2018 @05:25PM (#18490276)
What part of the indexing isn’t allowed, is it the scraping or the publishing of the scraped material? Or something else such as a “use” of the material?

Example: Does it matter whether the data is ever stored? Say I make a listing notifier that pings people when a house is listed in their desired area. I’d need to crawl real estate listings and but I’d never store or republish any content from the site.

Re:Web scraping case fails under Supreme Court's Dastar doct... by bigiain on Monday November 19, 2018 @06:24PM (#18490620)
The hardline copyright view is that to "index" content from a website, you need to make a copy of that content in your indexing server, which is an action copyright law legally prohibits without an explicit exception. (The people who argue that somehow also seem to argue that publishing the webpage legally gives "explicit permission" for your web browser to make a copy, without also giving permission for your web scraping software to do so. Not sure how they get to _that_ position logically...)
Re:Web scraping case fails under Supreme Court's Dastar doct... by alkonaut on Tuesday November 20, 2018 @02:32AM (#18492356)
Yeah but to e.g. make an app that notifies people when a house is published in area X really doesn't need to index anything. I can walk through all the listings once per week and all that is in my database is the table of what user wants what notification. If the data on the site matches my notification table I send an email (or whatever) and after the weekly run is done I still have zero data copied or "indexed" from the target site.
Re:Web scraping case fails under Supreme Court's Dastar doct... by jmiserez on Tuesday November 20, 2018 @04:36AM (#18492733)
Makes sense from a technical standpoint, but it's the legal people you'll need to convince.
Re:Web scraping case fails under Supreme Court's Dastar doct... by jscissr on Tuesday November 20, 2018 @05:51AM (#18492963)
Actually, you still have a copy of the data, in memory.
Re:Web scraping case fails under Supreme Court's Dastar doct... by alkonaut on Tuesday November 20, 2018 @06:03AM (#18493010)
yes. And I suspect it's not legal. Which is why I'm wondering what the actual definition is. Because it can't be that I "keep" or "index" the data, nor that I "republish" the data.

It's actually that I "read" the data, regardless of my intention and regardless of what I intend to do with it. But taken to the extreme there has to be something that makes it legal for me as a consumer to scrape ONE item from their data, while at the same time making it illegal for a company to scrape ALL their data in this way.

The worst situation to be in is one where it's "legal until it isn't" i.e., when you actually cause a problem (bandwidth, revenue) you will be sued - and until then you no idea.

Re:Web scraping case fails under Supreme Court's Dastar doct... by bigiain on Tuesday November 20, 2018 @05:57PM (#18498637)
It's that you "copy" it. That's the "right" that copyright law extends to creators.

Technically (and it's complicated these days because copyright law was mostly written well before computers and the internet existed) any time you create a copy of someone else's creative work, you need an explicitly granted right to do so. You can _read_ a book, but you cannot write down all or part of what you just read. You can listen to a song, but you cannot perform all or part of that song (An Australian band called Men At Work lost a copyright case for a flute solo that contained 2 bars/11 notes of a 1930's folk tune "Kookaburra" who's copyright had been sold/bought in 2000.)

There are some explicit legal exceptions to requiring a license, most commonly people talk about "fair use", but that's again a "pre internet" body of law, and is extremely open to interpretation as to how it applies to digital copies in computers.

https://www.copyright.gov/fair-use/more-info.html

https://fairuse.stanford.edu/overview/fair-use/what-is-fair-...