News for hackers. Stuff that matters.
Re:Web Scraping in 2016 by minimaxir on Tuesday August 23, 2016 @02:34PM (#12346236)
> tl;dr: Scraping isn't bad / blackhat as long as you aren't affecting their service or business.

Analyzing data that you're not allowed to access gives you/your company a competitive advantage, which is affecting their service/business even if it's not posted/distributed publically.

Re:Web Scraping in 2016 by Nicksil on Tuesday August 23, 2016 @02:48PM (#12346384)
I don't follow your argument. How does one get their scraper access to data they would otherwise not be able to access through 'normal' browsing techniques?
Re:Web Scraping in 2016 by Raphmedia on Tuesday August 23, 2016 @02:58PM (#12346460)
Example I know of: You can scrape your competitor's Facebook pages since their creation and output nice graphs of which posts generated what kind of likes and suscriptions. This data is usually limited to the owner of the page.
Re:Web Scraping in 2016 by wongarsu on Tuesday August 23, 2016 @03:11PM (#12346577)
But I can get the same data with manual browsing. And I can just as well pay some chinese to collect the data and input it in my database.

Automated scraping is just a way to drastically reduce labor costs for information collection. Sure, it's a competitive advantage, but I think disallowing it or calling it unethical is a pretty big can of worms. Why is it ok if something is done by humans but not ok if a computer does it by himself?

Re:Web Scraping in 2016 by Trundle on Tuesday August 23, 2016 @11:03PM (#12349398)
If someone was to sit down and use paper + pencil + time to accomplish the same thing, would you still have issue with it? It's publicly available data. Should you also not watch your competitors television ads or walk in to their physical store and browse around?
Re:Web Scraping in 2016 by Raphmedia on Wednesday August 24, 2016 @09:54AM (#12351987)
The difference being that I can scrape the data of ALL my competitors since 2008, get the number of likes and comments of every single comment and automatically generate graphs of the data. Thousands of man hours in one minute.

This gives an unfair advantage to the tech-savvy "hackers". Facebook terms protects against this. Thus scraping it is disallowed.

I couldn't say if it would be moral or immoral to do this. Personally, I'm more concerned about the well being of poor scraper program that has to scrape through an entire decade of Facebook posts. Poor thing.

Re:Web Scraping in 2016 by charlesdm on Tuesday August 23, 2016 @04:49PM (#12347493)
This is called hustling. I love it. What's not to like?
Re:Web Scraping in 2016 by ori_b on Tuesday August 23, 2016 @10:25PM (#12349247)
By ignoring robots.txt and bypassing capchas.