Here is a demonstration of a software stack that can scale toward hardware limits without relying on a particular vendor https://www.slideshare.net/SeanChittenden/freebsd-vpc-introd.... This approaches 100G line rate for large packets which is what it was optimized for. I don't know PPS at low packet size but do know what would be required to optimize that use case and it could be done pretty quickly.

Interestingly, the ENA driver has #defines for speeds up to 400 Gbps.

My guess as to why EC2 instances are limited to 25 Gbps is that it's a matter of balancing overprovisioning and the need to avoid having a single instance eat too much of a rack's bandwidth. I don't know how much bandwidth they have going to each rack, but there's a limit to how much it makes sense to provision; if typical bandwidth is on the order of 10 Gbps per rack (say, 80 instances pushing 125 Mbps on average) then you might want to provision 200 Gbps/rack and limit each instance to 25 Gbps rather than provisioning 1 Tbps/rack and limiting each instance to 100 Gbps.

(Numbers above are completely invented; I don't have any internal knowledge of how Amazon's networks or datacenters are set up.)

Also, are you talking about Annapurna in it's pre-acquisition form or new one? AWS talks about new custom asics and multiple ARM SoCs on their Nitro system.

AWS aren’t alone in this, and actually do pretty darn well compared to their competition - we had a nightmarish time a few years back with exactly this with a VPS provider - half of every second the traffic to the memcached cluster would just stop. Turned out they’d set hard limits on packets/sec to avoid oversaturating the host, so the advertised Gbps interconnect was actually 50Mbps when you saturated the packet scheduler.

Do GCP, Azure, or any other cloud providers offer 100G networking?

AWS Nitro allows 5G/bit per flow. And then maxes out at 25G/bit. I know GCP does something similar.

Also, pretty sure that is false regarding Azure, they have a small availability of Infiniband, but, that is not on their general compute platform and has a narrow use case/many restrictions. Azure has had the worst networking performance from my experience and only had 10GbE NICs (it's been a while though)

"Azure is breaking the speed barrier in cloud connectivity. ExpressRoute Direct provides 100G connectivity for customers with extreme bandwidth needs. This is 10x faster than other clouds."

https://azure.microsoft.com/en-us/blog/azure-networking-fall...

Can sb confirm that? Have a useful case in mind.

https://www.sparkpost.com/blog/undocumented-limit-dns-aws/

This issue can easily get amplifier if you're using Kubernetes on AWS and some library that didn't cache on DNS on its own. Imagine you have a healthcheck every 3 seconds, do a bunch of DNS to its dependencies services, and a single server may have 10 pods.

As suggested, it's very likely they hit the connection tracking limitation: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-ne...

I've personally witnessed teams hit this specifically for DNS (usually for internal, where you have explicitly permitted src/dst).

From powerdns https://doc.powerdns.com/recursor/performance.html

I don't know, but wouldn't be surprised if connection tracked packets are more limited than packets that aren't tracked.

[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-ne...

Where are you getting these numbers? For what instance types? For what protocols? This is just wrong.

Do you have a more accurate dataset? What are your observations?

I'm assuming this affects loads of HN readers and I too am interested in what the facts are.

Side note: you may be getting downvoted because a source, or other details are lacking. I too get downvoted for posts that lack these details.

However, with that said, i have always found that calling your rep and asking about specific un documented limits is the fastest way to get to the bottom of per-instance/account/vpc/whatever limits.

Just as there are limits that can be changed if you agree, in writing, that you will be financially responsible for whatever the impact is (e.g. when you could tell them that you wanted spot price limits adjusted for you to be able to better bid above the scaling factors that were in place(not sure if this is still the case))

Some limits are global and cant be changed/negotiated, but other undocumented limits....

Connect a single server to a network and it's oversubscribed. That's a bit hyperbolic, but even some beefy networks can be seriously burdened by just a single server spamming UDP packets without some sort of QoS.. Especially if they are bypassing user space and using the kernel to just replicate a bunch of packets onto the wire :)

I'm probably a bit biased from having spent time setting up linux TC on xen hypervisors for this very reason; and I think we even settled at 50k pps for the per vm limit too..

Maybe random reallocation to a rack with newer switches will guarantee better baseline PPS.

Could be OVS with DPDK support, for example.

This is perhaps already being seen in a piecemeal fashion as people compare eg S3 storage prices with other companies' prices.

What can you do when your system needs more bandwidth, cpu, ram or any other kind of resources?

You can either scale vertically... which is not bad at beginning of a project, but sooner or later you will hit the ultimate limit.

Or

You can scale horizontally. Which means that you have enough nodes or instances to bypass those limits and make sure your projects grow well over time.

Netflix runs on EC2 and they probably generate billions of PPS from Amazon. For sure it's several millions PPS a d they seem to not hit the limits mentionned in the article.