News for hackers. Stuff that matters.
Re:I Don’t Need No Stinking API: Web Scraping For Fun and Pr... by level09 on Sunday December 09, 2012 @01:27AM (#4894033)
That is actually very simple, and you can even use a headless browser to execute javascript:

first install Xvfb and pyvirtualdisplay then try this snippet https://gist.github.com/4243582

selenium is great, it can even wait for ajax requests to finish (see WebDriverWait) ..

Re:I Don’t Need No Stinking API: Web Scraping For Fun and Pr... by RaSoJo on Sunday December 09, 2012 @02:52AM (#4894189)
Yippie...that worked. Thanks a ton for sharing that code snippet. Am unfortunately more of an analyst. Have only just started picking up coding for data analysis. JS scraping was one area that I always had difficulty with. Not anymore :D
Re:I Don’t Need No Stinking API: Web Scraping For Fun and Pr... by kanzure on Sunday December 09, 2012 @09:09AM (#4894770)
> first install Xvfb and pyvirtualdisplay

You really don't need xvfb anymore. Use xserver-xorg-video-dummy.