Bypassing website anti-scraping protections

Re:Bypassing website anti-scraping protections by jamra on Friday June 01, 2018 @08:57AM (#17204820)

That’s precisely why they don’t want you cheating the hard part and just storing the results. It makes sense to me. Work on your own machine learning if you want good results.

[ Parent | on HN ]

Re:Bypassing website anti-scraping protections by toomuchtodo on Friday June 01, 2018 @09:48AM (#17205243)

There is no such thing as cheating, only staying within boundaries that don't land you in jail or sued in your own jurisdiction. If you can get an edge by using Google's own data, do so.

[ Parent | on HN ]

Re:Bypassing website anti-scraping protections by lugg on Friday June 01, 2018 @10:41AM (#17205689)

Bing bing bing!

Er, ughm. I mean,

Ding ding ding!

[ Parent | on HN ]

Re:Bypassing website anti-scraping protections by anc84 on Friday June 01, 2018 @09:35AM (#17205123)

You could argue that Google should work on their own knowledge database instead of learning from other people's content and/or presenting other people's content in their own frontends (shopping etc)...

[ Parent | on HN ]

Re:Bypassing website anti-scraping protections by heavenlyblue on Friday June 01, 2018 @10:08AM (#17205433)

This is what Common Crawl does: http://commoncrawl.org/. I think more people should know about it.

[ Parent | on HN ]