News for hackers. Stuff that matters.
Re:Lessons learned scraping 100B product pages by rosha on Tuesday July 10, 2018 @04:41PM (#17501952)
Golang is a good choice too but in my experience its nothing compared to what you can do with Erlang Queue and Elixir. Regarding your question about proxycrawl, I do not know honestly, I tested the service for few days on some few millions per day and it was great too. I would say they are good for a very high volume, we are still using it, so that should be a good signal to try them.

There are no comments on this comment yet.