Re:Colly: Fast and Elegant Scraping Framework for Golang
by weberc2 on Thursday October 05, 2017 @11:59AM
(#15410205)
You'd need to parse and run the JavaScript, including a virtual DOM. You'd also need to support any browser events that the page depends on. It's conceptually simple, and getting support for 80-95% of cases is probably easily doable, but I imagine a long tail of cases where your browser emulator isn't quite compatible in an important way. I know Google and others do this to index the web, but in not sure about any existing open source projects.
[ Parent
| on HN ]
There are no comments on this comment yet.