Re:How to scrape anything on the web and not get caught
by lemagedurage on Sunday April 22, 2018 @06:13AM
(#16895515)
You pay more per IP than per server!? Why don't you get 3 DO instances then?
[ Parent
| on HN ]
Re:How to scrape anything on the web and not get caught
by chrismeller on Sunday April 22, 2018 @09:02AM
(#16895984)
I can't speak for that particular service, but in other similar ones I've looked at and used in the past, you're paying for a certain number of IPs _at a given time_.
So, for instance, they have a pool of servers that have 1000 IPs available. Your account allows connections to go out over 2 of those at a time. If something happens (like one gets banned by whatever service you're scraping), you can get a different set of 2 IPs and keep moving.
While you're still paying a relatively high price for what you're consuming (predominantly bandwidth in this case), you're paying for the flexibility.
Re:How to scrape anything on the web and not get caught
by wiradikusuma on Sunday April 22, 2018 @06:55AM
(#16895613)
The paid service has IPs from a lot of other countries not covered by DO :(