News for hackers. Stuff that matters.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by packetslave on Friday September 28, 2018 @01:27AM (#18091274)
One thing missing from this analysis is the networking cost. The EC2 instance comes with 14gb/sec of dedicated EBS bandwidth and 25gb/sec of network bandwidth. That implies a 40gb NIC, which will require a 40gb top of rack switch and enough cross-sectional bandwidth in the datacenter fabric to handle however many of these machines they plan for (at whatever oversubscription ratio, if any).

That's a pretty non-trivial amount of expense to setup a network with enough capacity to support a machine like this fi you want to do it yourself.

Re:EC2 Instances with 6, 9, and 12 TB of Memory by mmt on Friday September 28, 2018 @02:21AM (#18091410)
> That implies a 40gb NIC

It may imply that but only outright requires 40Gb/s total, which is provided by the "4-port 10GBase-T Ethernet SIOM (Intel® X550 controller)" of a SuperServer 7089P-TR4T.

The EBS bandwidth can be elided entirely by using local storage instead. Local storage bandwidth can be much cheaper than ethernet, especially considering how many NVMe ports are already included.

As 10GE is maybe $200/port (including SFPs), at scale, network cost doesn't "move the needle" on a $400k server.