News for hackers. Stuff that matters.

A comment on Ask HN: Advanced web crawling resources?. Parent comment

Re:Ask HN: Advanced web crawling resources? by iagovar on Sunday April 30, 2017 @05:21AM (#14230767)
I use couchdb with replication and postgreSQL as data warehouse.

Anyway Im a noob, but reading here and there is what I decided to use.

For scraping Im using scrapy + selenium and a modified js script that uses chrome (webscraper.io).

[ Parent | on HN ]
Re:Ask HN: Advanced web crawling resources? by z3t4 on Monday May 01, 2017 @06:29AM (#14236589)
i would just make a naive implementation, instead of searching for the optimal tools and solutions.
[ Parent | on HN ]