News for hackers. Stuff that matters.
Re:AWS X1 instances – 1.9 TB of memory by scaleout1 on Wednesday May 18, 2016 @01:17PM (#11723618)
When you suddenly realize that your "big" data is not really that big!. Who needs a Hadoop/Spark cluster when you can run one of these bad boys
Re:AWS X1 instances – 1.9 TB of memory by tracker1 on Wednesday May 18, 2016 @01:31PM (#11723802)
That was kind of my thought as well... I worked on a small-mid sized classifieds site (about 10-12 unique visitors a month on average) and even then the core dataset was about 8-10GB, with some log-like data hitting around 4-5GB/month. This is freakishly huge. I don't know enough about different platforms to even digest how well you can even utilize that much memory. Though it would be a first to genuinely have way more hardware than you'll likely ever need for something.

IIRC, the images for the site were closer to 7-8TB, but I don't know how typical that is for other types of sites, and caching every image on the site in memory is pretty impractical... just the same... damn.

Re:AWS X1 instances – 1.9 TB of memory by sciurus on Wednesday May 18, 2016 @04:49PM (#11725819)
I think you're missing a unit. 10-12 thousand? million?
Re:AWS X1 instances – 1.9 TB of memory by tracker1 on Thursday May 19, 2016 @12:27PM (#11731374)
million... lol
Re:AWS X1 instances – 1.9 TB of memory by samstave on Wednesday May 18, 2016 @01:44PM (#11723953)
Heh, but I wonder what the default per account limits are on launching these... prolly (1) per account.
Re:AWS X1 instances – 1.9 TB of memory by cwyers on Wednesday May 18, 2016 @02:11PM (#11724262)
Why would they put any kind of a limit on it?
Re:AWS X1 instances – 1.9 TB of memory by samstave on Wednesday May 18, 2016 @02:23PM (#11724393)
All AWS accounts have a "limits" which have the default limits as to how many instances that you could launch in that region.

The reason is so if you fuck up a scaling script for example you can't launch 1000 machines and take all the capacity and then bitch that you won't pay for it.

It's a stop gap.

However, aside from the hard limit of 100 S3 buckets, all other limits are configurable at the request of your AWS rep

Re:AWS X1 instances – 1.9 TB of memory by geofft on Wednesday May 18, 2016 @03:09PM (#11724833)
It looks like that hard limit became a soft limit in August:

https://aws.amazon.com/about-aws/whats-new/2015/08/amazon-s3...

Re:AWS X1 instances – 1.9 TB of memory by samstave on Wednesday May 18, 2016 @03:40PM (#11725146)
Hmm... I didn't know this. The last time I asked they said it would never happen. I was told the original reason was that all buckets had to have a unique name
Re:AWS X1 instances – 1.9 TB of memory by admiun on Wednesday May 18, 2016 @04:18PM (#11725523)
I thought that too but I successfully raised our limit after they made it a soft limit, so give it a shot!
Re:AWS X1 instances – 1.9 TB of memory by slaman on Wednesday May 18, 2016 @03:15PM (#11724883)
because they can only put these into racks so fast
Re:AWS X1 instances – 1.9 TB of memory by nucleardog on Wednesday May 18, 2016 @04:20PM (#11725540)
And to prevent a run-away script from suddenly spooling up thirty of them. Besides issues with their hardware capacity, they're generally pretty good about refunding mistakes like that, so they're eating the cost...