News for hackers. Stuff that matters.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by packetslave on Thursday September 27, 2018 @05:46PM (#18089067)
Also gives you 224 physical CPU cores (448 logical cores).

Requires a 3-year reservation, and "the effective hourly rate for the All Upfront 3-Year Reservation for a u-12tb1.metal Dedicated Host in the US East (N. Virginia) Region is $30.539 per hour."

Works out to $267,512.88 per year, or $802,538.64 for the 3-year term. I wonder how that compares to building your own on-premise host with that much RAM (obviously, there's operational costs to consider as well).

Also, don't miss the last line: "We’re not stopping at 12 TiB, and are planning to launch instances with 18 TiB and 24 TiB of memory in 2019."

Re:EC2 Instances with 6, 9, and 12 TB of Memory by mmt on Thursday September 27, 2018 @08:29PM (#18090058)
> $802,538.64 for the 3-year term. I wonder how that compares to building your own on-premise host with that much RAM

This was discussed a couple days ago in a different thread [1], when 4TB was the EC2 limit.

$400k for the current-gen (224 cores) or, as a sibling comment [2] notes, $250k for the previous CPU generation (which can use twice as many DIMMs of half the density and only 192 cores of that).

> obviously, there's operational costs to consider as well

We can estimate an upper bound on this, given that the current-gen system has N+2 5x1600W 96% efficient PSUs, so 5kW max. If you're paying a colo $.50/kWh, that's another $66k over 3 years worst case.

Realistically, though, CPUs with a max TDP of 1640W, DIMMs (generously) 700W, leaves plenty of room for fans, SSDs and other overhead before getting to 3kW or $40k, and that's still assuming running full-bore the whole time.

There are obviously also ancillary costs to AWS, such as data transfer and EBS.

If you're already running your own hardware, it sure does look more attractive to pay $400k now and $40k over 3 years than to pay AWS $800k All Up Front (now). If not, perhaps it's more attractive to hand-wave away that $360k as saving the "hassle" of hiring someone who knows how to run equipment in a datacenter.

[1] https://news.ycombinator.com/item?id=18041486

[2] https://news.ycombinator.com/item?id=18089267

Re:EC2 Instances with 6, 9, and 12 TB of Memory by packetslave on Friday September 28, 2018 @01:27AM (#18091274)
One thing missing from this analysis is the networking cost. The EC2 instance comes with 14gb/sec of dedicated EBS bandwidth and 25gb/sec of network bandwidth. That implies a 40gb NIC, which will require a 40gb top of rack switch and enough cross-sectional bandwidth in the datacenter fabric to handle however many of these machines they plan for (at whatever oversubscription ratio, if any).

That's a pretty non-trivial amount of expense to setup a network with enough capacity to support a machine like this fi you want to do it yourself.

Re:EC2 Instances with 6, 9, and 12 TB of Memory by mmt on Friday September 28, 2018 @02:21AM (#18091410)
> That implies a 40gb NIC

It may imply that but only outright requires 40Gb/s total, which is provided by the "4-port 10GBase-T Ethernet SIOM (Intel® X550 controller)" of a SuperServer 7089P-TR4T.

The EBS bandwidth can be elided entirely by using local storage instead. Local storage bandwidth can be much cheaper than ethernet, especially considering how many NVMe ports are already included.

As 10GE is maybe $200/port (including SFPs), at scale, network cost doesn't "move the needle" on a $400k server.

Re:EC2 Instances with 6, 9, and 12 TB of Memory by mmt on Saturday September 29, 2018 @12:31AM (#18098723)
Re:EC2 Instances with 6, 9, and 12 TB of Memory by danbruc on Thursday September 27, 2018 @06:12PM (#18089267)
This [1] is the closest thing I could quickly find, so I guess we are talking on the order of $250k to own one.

[1] https://www.thinkmate.com/system/superserver-7088b-tr4ft

Re:EC2 Instances with 6, 9, and 12 TB of Memory by _wmd on Thursday September 27, 2018 @06:27PM (#18089378)
At this size calculating power consumption is probably important, it's definitely no longer insignificant
Re:EC2 Instances with 6, 9, and 12 TB of Memory by qaq on Thursday September 27, 2018 @06:33PM (#18089429)
even at 1K per month it would add 36K doesn't change the equation much
Re:EC2 Instances with 6, 9, and 12 TB of Memory by NedIsakoff on Thursday September 27, 2018 @06:56PM (#18089613)
That's 8 nodes with 1 CPU each.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by danbruc on Thursday September 27, 2018 @07:25PM (#18089765)
There is one daughter board per socket but it is a single system.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by sehugg on Thursday September 27, 2018 @06:32PM (#18089420)
I could grep the heck out of some log files on that.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by TheDong on Thursday September 27, 2018 @06:36PM (#18089457)
I know you're joking, but that operation is often disk-io bound, not cpu or memory bound... and the disk is still EBS, you'll not get much better performance on that than on an m5.xlarge I bet, unless you use a grep regex that needs to gobble lots of data into memory as part of the search.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by justinsaccount on Thursday September 27, 2018 @06:42PM (#18089508)
> that operation is often disk-io bound

the first time.. 12TB can cache a lot of logs.

Re:EC2 Instances with 6, 9, and 12 TB of Memory by oconnor663 on Thursday September 27, 2018 @06:44PM (#18089521)
It's disk-bound the first time you run grep, but the second time you run it the log file is going to be in memory, if you have enough memory.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by ohazi on Thursday September 27, 2018 @06:39PM (#18089477)
With that much memory, a ramdisk would seem completely reasonable.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by KindOne on Thursday September 27, 2018 @06:40PM (#18089493)
tmpfs would solve the disk-io
Re:EC2 Instances with 6, 9, and 12 TB of Memory by glandium on Thursday September 27, 2018 @06:54PM (#18089596)
Which brings the interesting question: what kind of entity would pay Amazon $800K upfront for 3 years of using a single (massive) machine?
Re:EC2 Instances with 6, 9, and 12 TB of Memory by detaro on Thursday September 27, 2018 @07:56PM (#18089903)
Large enterprises that are willing to use AWS. Not an accident they talk about SAP certification in the announcement.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by Aeolun on Thursday September 27, 2018 @08:45PM (#18090133)
Honestly, 1M transactions per hour on that monster sounds a bit low.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by glandium on Thursday September 27, 2018 @09:41PM (#18090346)
This feels like it circles back to https://news.ycombinator.com/item?id=18012334
Re:EC2 Instances with 6, 9, and 12 TB of Memory by nunez on Thursday September 27, 2018 @07:04PM (#18089653)
The government
Re:EC2 Instances with 6, 9, and 12 TB of Memory by nikkwong on Thursday September 27, 2018 @06:13PM (#18089273)
Crazy. Is the use case for these types of instances pretty much designated for ML? Why else would anyone need these?
Re:EC2 Instances with 6, 9, and 12 TB of Memory by lurker456 on Thursday September 27, 2018 @06:33PM (#18089428)
HANA in memory database that can serve both OLTP and OLAP workloads with sub second response times. The cost of the server is dwarfed by the licence cost.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by regularfry on Thursday September 27, 2018 @06:23PM (#18089347)
Any time you might think "I need a Hadoop cluster for this!" is now less likely to be correct.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by lelf on Thursday September 27, 2018 @06:20PM (#18089334)
From the page:

>> SAP HANA in Minutes

> The EC2 High Memory instances are certified by SAP for ⟨…⟩

[SAP HANA keeps everything in RAM.]

Re:EC2 Instances with 6, 9, and 12 TB of Memory by danbruc on Thursday September 27, 2018 @06:16PM (#18089298)
I don't think machine learning would be a good use case for such a machine, you probably want GPUs or TPUs for that.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by trynumber9 on Thursday September 27, 2018 @06:19PM (#18089325)
Database with the entire data set in memory?
Re:EC2 Instances with 6, 9, and 12 TB of Memory by abraham_lincoln on Thursday September 27, 2018 @06:22PM (#18089345)
My redis is bigger than your redis?
Re:EC2 Instances with 6, 9, and 12 TB of Memory by Avamander on Thursday September 27, 2018 @06:39PM (#18089481)
This sentence seems especially peculiar when you take into account that the word "redis" means "radish" in Estonian.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by plumeria on Thursday September 27, 2018 @07:29PM (#18089791)
Funny that Redis has en edit distance of 2 with certain word... and also rhymes with it.
Re:EC2 Instances with 6, 9, and 12 TB of Memory by aquadrop on Thursday September 27, 2018 @06:24PM (#18089358)
Database server?