Many of them have managed object storage services as well. OVH[1] and Linode[2], scaleway[3] have them, that should scale for most use cases and are S3 compatible APIs

Also Azure and Linode, Scaleway Backblaze and others are part of Cloudflare bandwidth alliance [4] so there shouldn't be egress fees between the two.

It is really only AWS which is a problem, you don't need this setup with any other provider.

[1] https://www.ovhcloud.com/en/public-cloud/object-storage/

[2] https://www.linode.com/products/object-storage/

[3] https://www.scaleway.com/en/object-storage/

[4] https://www.cloudflare.com/bandwidth-alliance/

You can use block storage for scalable disk size: https://www.linode.com/products/block-storage/

But then you're right back to the cloud billing problem, right?

That's right, auto-scaling comes with this problem, but at least you removed one extra service/point of failure.

That's assuming you trust Linode block storage as much as you trust S3.

Trust in what sense? Uptime, security, privacy? I am not sure whether I can say I trust one or another, by personally I had a good experience with VPS/dedicated server providers, more than with cloud (AWS/GCP).

Out of curiosity I tried to look up their pricing and the first thing I am greeted with when launching their price calculator is "you must allow functional cookies".

I disabled all shields for their side and still the same thing. Waste of time

I personally never used Linode and can not recommend nor talk against it, I was just pointing out that if you want scalable solutions AWS is not the only answer.

If your cache is much smaller than the data, it will be ineffective, unless you think everyone keeps downloading the same tiny subset of files. That last assumption works for web content (e.g. newest articles see more hits) but probably not for data.