Re:Designing a scalable API on AWS spot instances
by fulafel on Thursday July 23, 2020 @10:27AM
(#23927689)
There's options outside SaaS object storage though, eg running your own MinIO or Ceph, + DAS (or SAN) at your colo. Yes it's some hassle but if you have significant expenses from SaaS it might be worth checking out.
Re:Designing a scalable API on AWS spot instances
by dijit on Thursday July 23, 2020 @10:17AM
(#23927589)
AWS S3 buckets have a limit of 5gb/s access to a single bucket.
Unless that limitation has changed in the last couple of years; I can easily make a system beat that, if that's the requirement.. and ultimately it does come down to understanding requirements. :\
I think generally people forget that cloud is just computers too, it's really not anything special, and amazon/google/microsoft are solving the general case (and, doing so well, actually) but this comes at a high premium.
Re:Designing a scalable API on AWS spot instances
by duggan on Thursday July 23, 2020 @11:36AM
(#23928433)
It's 25Gbps now[1]. I have a rule of thumb: if I haven't refreshed something I think I know about AWS for more than half a year, it's probably out of date.
[1]: https://aws.amazon.com/premiumsupport/knowledge-center/s3-ma...
Re:Designing a scalable API on AWS spot instances
by saddlerustle on Thursday July 23, 2020 @01:10PM
(#23929573)
That the throughput available to a single EC2 instance. S3 as a whole will do hundreds of Gbps no problem.
Re:Designing a scalable API on AWS spot instances
by speedgoose on Thursday July 23, 2020 @09:11AM
(#23926963)
That's not a very common use case though. Most companies don't need to DDOS their cloud provider.
Re:Designing a scalable API on AWS spot instances
by jrockway on Thursday July 23, 2020 @12:28PM
(#23929081)
25Gbps is 250 cable Internet users downloading the release of your new software. What you call a DDOS is actually a woefully underwhelming instantaneous transfer rate.
Re:Designing a scalable API on AWS spot instances
by Epixors on Thursday July 23, 2020 @01:54PM
(#23930119)
If you're distributing releases (or anything, really) users should not have to go directly to your object store...
Re:Designing a scalable API on AWS spot instances
by speedgoose on Thursday July 23, 2020 @01:36PM
(#23929891)
You should put your objects storage behind a CDN if you expect that many downloads. That's much cheaper and faster.
Lots of providers claim to offer object storage, but try hitting them from couple thousand cores and they all tend to immediately fall over.