Re:RoboBrowser: Your friendly neighborhood web scraper
by enibundo on Thursday May 12, 2016 @03:35AM
(#11681934)
Last time I needed something like this I used selenium.
And I use requests the rest of the time.
[ Parent
| on HN ]
Re:RoboBrowser: Your friendly neighborhood web scraper
by hendler on Thursday May 12, 2016 @03:39AM
(#11681949)
Same here. I use python selenium to hit a selenium server for some speed improvements. Chrome/Firefox/Phantomjs, and can inject custom javascript over the pages. Still about 10 seconds to load, render, process a page.