News for hackers. Stuff that matters.
Re:Lessons learned scraping 100B product pages by pdimitar on Tuesday July 10, 2018 @03:07PM (#17501163)
Then I think we have a misunderstanding of terms. To me "multicore" == "single process, many threads". Apologies for the confusion.

It seems that now we are both on the same page. Single process & many threads are problematic for GIL languages and that's why I gave up using Ruby for scrapers. GIL languages can work very well for the URL downloading part though.

There are no comments on this comment yet.