News for hackers. Stuff that matters.
Re:Ask HN: What are best tools for web scraping? by sharmi on Tuesday November 14, 2017 @09:11AM (#15695073)
Nope. It is very specifically tailored to crawling. If you just need something distributed why not check out RQ [0], Gearman [1] or Celery [2]? RQ and Celery are python specific.

[0] : http://python-rq.org/docs/

[1] : http://gearman.org/

[2] : http://docs.celeryproject.org

There are no comments on this comment yet.