Re:Colly: Fast and Elegant Scraping Framework for Golang
by asciimoo on Thursday October 05, 2017 @10:08AM
(#15409268)
Scraping JS-only sites is also possible without a headless browser, but requires a bit more debugging of the internal structure of these sites.
Most of JS-only websites have API endpoints with JSON responses, which can make scraping more reliable than parsing custom (and sometimes invalid) HTML.
The drawback of headless browser based scraping is that it requires significant amount of cpu time and memory compared to "static" scraping frameworks.
[ Parent
| on HN ]
There are no comments on this comment yet.