News for hackers. Stuff that matters.
Re:Amazon2csv: Amazon products scraper to CSV (no API token ... by zeusk on Sunday June 10, 2018 @11:15PM (#17281474)
why aren't Larry and Sergey behind bars, then? Scraping publicly available information is far from illegal.

Also, Interestingly only Alibaba's bots are completely blocked from crawling: https://www.amazon.com/robots.txt

Re:Amazon2csv: Amazon products scraper to CSV (no API token ... by stef25 on Monday June 11, 2018 @04:33AM (#17282718)
> Scraping publicly available information is far from illegal.

The scraping itself may not be (although I'm pretty sure here in Belgium there is a law against collecting other people's data), but what you do with it may not be legal.

You could make a case for making any kind of profit generated from scraping data illegal. Don't get me wrong, I love scraping things myself.

Also find it amazing there are companies out there like Crawlera that can do serious scraping work and openly flaunt deploying tech to get around whatever scraping blockers are out there.

Re:Amazon2csv: Amazon products scraper to CSV (no API token ... by cosmie on Monday June 11, 2018 @02:46PM (#17287178)
>

LinkedIn had multiple layers of scraping detection systems deployed, and went to significant efforts to block their data from getting scraped[1].

Last year, they were ordered by a Federal court explicitly to allow scraping of content and remove systems that were designed to impede and block scraping efforts[2].

There's no clear law (in the US) directly aimed at scraping, and repurposing anti-hacking laws brought up the murky definition of what is unauthorized access. If a judge clarifies explicitly that scraping is not unauthorized access (which happened in this case, although needs to stand up to appeal[3]), then entities that are interested in preventing scraping have lost one of their core legal underpinnings. It demonstrates why companies like Crawlera have been able to flaunt the type of serious scraping work they do, because it's hard to bully people with a legal argument that has been debunked and affirmed as debunked on appeal. So it's better to avoid the risk of setting that precedence entirely until you can't avoid it.

[1] https://techcrunch.com/2016/08/15/linkedin-sues-scrapers/

[2] https://www.reuters.com/article/us-microsoft-linkedin-ruling...

[3] https://www.courthousenews.com/linkedin-takes-data-scraping-...

Re:Amazon2csv: Amazon products scraper to CSV (no API token ... by kull on Monday June 11, 2018 @01:41AM (#17282093)
Check amazon api T&C, also try to do the same with Craigslist and see how long you they will let you do it. scraping data is always a shady business if you do it without a permission of content owner
Re:Amazon2csv: Amazon products scraper to CSV (no API token ... by zeusk on Monday June 11, 2018 @02:54AM (#17282338)
It is anything but shady. They can send you a C&D or file a suit and seek injunction but there is no way they can get you in trouble with the law for scraping publicly available data.
Re:Amazon2csv: Amazon products scraper to CSV (no API token ... by gsich on Monday June 11, 2018 @06:39AM (#17283173)
I don't need to check the T&C to view the page.
Re:Amazon2csv: Amazon products scraper to CSV (no API token ... by rmwaite on Monday June 11, 2018 @02:44AM (#17282292)
If you’re scraping.. you’re not using the API.