News for hackers. Stuff that matters.
Re:Osmosis: Web scraper for Node.js by risyasin on Sunday April 05, 2015 @06:17PM (#9325522)
Well, you are right about nodejs land & what it means being lightweight in that land. Frankly it looks like hell when you have to deal with not only packages also with their semantic versions. but being another nodejs crawler library owner (Arachnod: Web Crawler for Node.js https://www.npmjs.com/package/arachnod), i can assure you that Developer has a point about being lightweight & no dependency of his package. When i had started to code a crawler with nodejs i had to deal with many problems (i believe amount of problems may be less for other common languages)

Also i haven't tried it for a long shot for example to make it work more than millions of webpages, but "memory leak free" is a really strong claim which has to be tested first.

Re:Osmosis: Web scraper for Node.js by watson on Sunday April 05, 2015 @06:36PM (#9325572)
I'm interested to hear what you think is problematic when building a web crawler in regards to dependencies? Is it specific to the DOM parsing?
Re:Osmosis: Web scraper for Node.js by fapjacks on Sunday April 05, 2015 @07:37PM (#9325711)
I've toyed with web scraping in Node, and the answer is definitely parsing.