News for hackers. Stuff that matters.
Re:Running a scraping platform at Google Cloud for as little... by chumanak on Thursday October 03, 2019 @07:10AM (#21145665)
great article, the problem I found in those cases is that some pages like immobilienscout24 sometimes take a lot to load, and that increases the cost. Also in my case I found some websites (not immobilienscout24) blocking my requests, so I ended up implementing proxycrawl to unblock the requests, that allowed to bypass the blocks of google cloud but still I faced the loading times problem of some of the websites

There are no comments on this comment yet.