Re:Running a scraping platform at Google Cloud for as little...
by chumanak on Thursday October 03, 2019 @07:10AM
(#21145665)
great article, the problem I found in those cases is that some pages like immobilienscout24 sometimes take a lot to load, and that increases the cost. Also in my case I found some websites (not immobilienscout24) blocking my requests, so I ended up implementing proxycrawl to unblock the requests, that allowed to bypass the blocks of google cloud but still I faced the loading times problem of some of the websites
[ Parent
| on HN ]
There are no comments on this comment yet.