Re:Web Scraping in 2016
by fake-name on Tuesday August 23, 2016 @03:28PM
(#12346735)
None of the sites I'm scraping do that, mostly.
I'm not scraping high value sites like that (I mostly target amateur original content). It's not really of interest to other businesses. As such, I tend to just run into things like normal cloud-flare wrapped sites, and one place that tried to detect bots and return intentionally garbled data.
If I run into that sort of thing, I guess we'll see.
I'm not scraping high value sites like that (I mostly target amateur original content). It's not really of interest to other businesses. As such, I tend to just run into things like normal cloud-flare wrapped sites, and one place that tried to detect bots and return intentionally garbled data.
If I run into that sort of thing, I guess we'll see.