Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu...
by zbjornson on Tuesday January 05, 2016 @05:54PM
(#10846743)
That's one of many data sets on the server, so unfortunately we can't keep them all in memory at once. :(
[ Parent
| on HN ]
Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu...
by ChuckMcM on Tuesday January 05, 2016 @10:22PM
(#10848240)
Lets assume when you're saying "cpu" when you mean "core" and your typical server class machine has 24 of those. A 1000 "cpus" is 41 machines, if they each donate 32GB to the cause[1] that is 1.3TB worth of data which is only a few microseconds away from any core.
I'm not sure why anyone would build a server with less than 96GB on it these days, so its not at all unreasonable. Now your service provider my jerk you around but you can run two racks of machines (48 machines) in a data center with specs like that for about $25K/month (including dual gigabit network pipes to your favorite IP transit provider) So it isn't even all that huge of an investment.
[1] Consider your typical 'memcached' type service where data is named as a function of IP and offset.