Re:Colly: Fast and Elegant Scraping Framework for Golang
by Walkman on Thursday October 05, 2017 @06:59PM
(#15413209)
I totally agree! I like Go but this is not a field I would ever use it. Parsing the site-s will never will be the bottleneck, but following the HTML changes or making it work for multiple sites is... When it takes maybe even seconds to download a page, a couple of millisecond performance advantage of Go doesn't matter at all.
Here, unless you're parsing a large amount of already downloaded files (a website dump, [re]parsing of a long-standing archive etc.), you're not going to get a huge benefit from using a fast parser, because network is going to be the challenging factor.
Keep that in mind.