Re:Colly: Fast and Elegant Scraping Framework for Golang
by stcredzero on Thursday October 05, 2017 @04:26PM
(#15412325)
It basically runs headless chrome with a JavaScript REPL console you can use to interact with the session.
That looks cool! Would I be able to run Node scripts?
Re:Colly: Fast and Elegant Scraping Framework for Golang
by simlevesque on Thursday October 05, 2017 @05:20PM
(#15412677)
> Would I be able to run Node scripts?
No. Since you can't run Node scripts on Chrome, the same is true for Chrome headless.
For that stuff, as of a few months ago, you can use Chrome headless. I wrote a couple go packages to make that easy. It basically runs headless chrome with a JavaScript REPL console you can use to interact with the session. https://GitHub.com/integrii/headlessChrome
I was also able to smash my while scraper bot into a docker container after working around a couple bugs.