News for hackers. Stuff that matters.
Re:Preventing Site Scraping by ceejayoz on Thursday May 17, 2012 @05:11PM (#3989119)
> By loading the paginated data through javascript without a page reload, this significantly complicates the job for a lot of scrapers out there. Google only recently itself started parsing javascript on page. There is little disadvantage to reloading the data like this.

Well, unless you're visually impaired and using a screen reader... and it doesn't really complicate things for any halfway dedicated scraper, as your AJAX pagination requests probably follow the same predictable pattern as the non-AJAX ones would've.

Re:Preventing Site Scraping by jacobr on Thursday May 17, 2012 @05:24PM (#3989183)
Do you have any real-world examples of commonly used screen readers that can't handle JavaScript? Screen readers gets its content from the DOM in a browser, so if the browser is able to put it in the DOM, it should be available to the screen reader.

98.4% of screen reader users in a 2010 survey (http://webaim.org/projects/screenreadersurvey3/#javascript) had JavaScript enabled.

Re:Preventing Site Scraping by TeMPOraL on Thursday May 17, 2012 @05:39PM (#3989253)
And if a screen reader can see the data, so can the scrapper.
Re:Preventing Site Scraping by joshu on Thursday May 17, 2012 @05:14PM (#3989133)
AJAXification could make it much easier. GET -> json import.
Re:Preventing Site Scraping by paulgb on Thursday May 17, 2012 @06:45PM (#3989521)
Exactly what I was thinking. Heck, I've scraped using fake AJAX requests even when a legitimate API was available, simply because opening up firebug was faster than reading the API docs.
Re:Preventing Site Scraping by jamesaguilar on Thursday May 17, 2012 @07:01PM (#3989568)
This is exactly what I used for my project to scrape Path of Exile item data.
Re:Preventing Site Scraping by joshu on Thursday May 17, 2012 @07:26PM (#3989646)
Ha! I did the same for KoL once upon a time.
Re:Preventing Site Scraping by jumpbug on Thursday May 17, 2012 @05:14PM (#3989135)
That is why I suggest using as many of these in combination as possible. A lot of your basic scraping scripts won't have this functionality built in.