Or use it to get stuff on the internet that they don't want to get directly themselves. Once upon a time I accidentally enabled open proxying on the web server on my Linux box. This was before smart phones and tablets and such, when my Linux box was the only network client I had, and so I didn't have a NAT router. Just modem straight to Linux box.

One day I checked my Apache logs to see if my extremely low traffic web site had any visitors.

First impression: "What the hell is all this traffic? This is orders of magnitude more than I should have!".

Second impression: "Why the hell is someone trying to get horse_fucking.mov from my server?".

Third impression: "That didn't give a 404. Why is my server actually providing horse_fucking.mov?".

(No, the orders of magnitude extra traffic were not all people getting horse_fucking.mov. There was a whole plethora of bestiality titles, plus a lot of other porn that you would not want to fetch directly from your own IP address either out of embarrassment if you got caught or because it was almost certainly illegal).

That was the first hand experience that confirmed my suspicions of what those “free proxies” really are.

    Calling: https://nsa.gov, headers = BOMBTHEMOSQUE -> 911
    Calling: https://fbi.net
    Calling: file:///bin/sh
    Calling: file:///etc/passwd
    Calling: http://127.0.0.1:80

That's when I realized from the scrolling log messages that even residental ADSL was already under heavy probing from around the world.

I decided I didn't have the time or the stomach to try to deal with that at home, and there wasn't much I could do, so I disabled the logging.

That's a good point, another one is people using your proxy to attack other servers.

I find the proxy area to be interesting but something I'm unlikely to use for business, you are quoting some examples that made me laugh, and I'd certainly laugh if it happens to me, but I know I will remove the public access, if I were involved in business with this, the story would be different.

Thanks for sharing.

I'm not that worried about websites banning the Pi as the traffic is really low, on concurrent calls it's unlikely to be a problem as I don't expect the Pi to query several websites in parallel.

Certainly it's something worth implementing if I ever start getting more traffic.

Thanks for sharing.

For this article, does the OP feel they need the proxy due to scraping rapidly?

The actual problem is that some websites block IPs from known cloud providers, so, it doesn't matter which headers you use because what's blacklisted is the IP address.

In the post I detailed that I run a service (https://cazadescuentos.net) which is a browser extension to look for discounts.

Sometimes, such service needs to look for the actual discounts, it queries the actual stores from the server that's on the cloud, but some stores doesn't let it go.

So, I mounted a Pi at home that resolves only those problematic queries, I expose the Pi at home to my cloud server by using a SSH tunnel.

Mainly the custom-made proxy allows scaling the app to several pies if I ever need to.

Just ssh -D 12345 raspberry and then just use localhost:12345 as the scrapers SOCKS5 proxy. No need to involve a http proxy.

EDIT: AAh, ssh port is not accessible from outside and the raspi does a reverse connect.

A reason to writing the custom-made proxy is mostly to evolve it supporting several pies, the tunnel being the simpler workaround for now.

Also, I'm a coder and wanted to try getting Scala running on my old pi.

https://www.harding.motd.ca/autossh/

Do you have any experiences with autossh/ssh to share?

Also making a backup image of the SD card once configured and using a NAS or external disk for important info helps mitigate the data loss risk, and gives you a quick path to restore onto a fresh SD.

With that said, I have a few raspberry pis that have been running for years without SD failures.

Also, make sure swap is not enabled on the SD

What is total RSS of all your proxy processes?

    $ free -h
                  total        used        free      shared  buff/cache   available
    Mem:          432Mi       194Mi        26Mi        20Mi       211Mi       164Mi
    Swap:          99Mi       2.0Mi        97Mi



    $ top
    Tasks:  69 total,   1 running,  68 sleeping,   0 stopped,   0 zombie
    %Cpu(s):  7.3 us,  4.3 sy,  0.0 ni, 88.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
    MiB Mem :    432.4 total,     26.2 free,    194.7 used,    211.5 buff/cache
    MiB Swap:    100.0 total,     98.0 free,      2.0 used.    164.7 avail Mem

      PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
     2915 pi        20   0  299.9m 173.8m  13.6m S   1.6  40.2 447:45.82 java
    32009 pi        20   0    7.5m   2.6m   2.2m R   1.0   0.6   0:00.31 top
    31991 pi        20   0   11.9m   3.8m   3.0m S   0.3   0.9   0:00.10 sshd

You can gain most (not all) of that 64MB back for free, simply by running `echo gpu_mem=1 | sudo tee -a /boot/config.txt && sudo reboot`

(Pretty sure that the minimum is actually more than 1, but also that it will accept that and set it to the minimum... check the docs if you need to be sure)

I'm still thinking to use several Pies and it's simpler to evolve the custom-made proxy to support that, if you see the Github repo, it's pretty simple.

Last, getting a Scala project to run on the old Pi was a nice experiment worth trying.