A Raspberry Pi as a decent residential proxy

Re:A Raspberry Pi as a decent residential proxy by city41 on Monday June 08, 2020 @09:20PM (#23462643)

Myself and the person I was replying to ran our scrapers from our homes. One benefit being the IP address doesn’t get flagged. Curious why scraping from your home network doesn’t work for you?

[ Parent | on HN ]

Re:A Raspberry Pi as a decent residential proxy by AlexITC on Monday June 08, 2020 @09:24PM (#23462666)

Actually, that's what I'm doing.

In the post I detailed that I run a service (https://cazadescuentos.net) which is a browser extension to look for discounts.

Sometimes, such service needs to look for the actual discounts, it queries the actual stores from the server that's on the cloud, but some stores doesn't let it go.

So, I mounted a Pi at home that resolves only those problematic queries, I expose the Pi at home to my cloud server by using a SSH tunnel.

[ Parent | on HN ]

Re:A Raspberry Pi as a decent residential proxy by city41 on Monday June 08, 2020 @09:52PM (#23462844)

Yeah I saw that in your article. I’m wondering why bother with the cloud server? Why not scrape all stores from home?

[ Parent | on HN ]

Re:A Raspberry Pi as a decent residential proxy by AlexITC on Monday June 08, 2020 @10:23PM (#23463016)

The latency with the Pi is too high to handle all stores, I could easily do that if I was using a more powerful device.

[ Parent | on HN ]

Re:A Raspberry Pi as a decent residential proxy by city41 on Monday June 08, 2020 @10:34PM (#23463069)

Ah I see, cool thanks for clarifying. That does make sense.

[ Parent | on HN ]