Memory support for versions of Windows:
User-mode process Kernel-mode
address space address space
(process bit-ness)
32-bit 64-bit all
Server Operating Systems:
Windows Server 2008 R2 4 GB* 2048 GB 2048 GB
Windows Server 2012 4 GB* 4096 GB 4096 GB
Windows Server 2012 R2 4 GB* 131072 GB 131072 GB
Client Operating Systems:
Windows 7 (x86) 3 GB* n/a 2 GB
Windows 7 (64-bit) 4 GB* 192 GB 192 GB
Windows 8 (64-bit) 4 GB* 512 GB 512 GB
Windows 8.1 (64-bit) 4 GB* 131072 GB 131072 GB
* requires IMAGE_FILE_LARGE_ADDRESS_AWARE flag, otherwise 2GB
** All units use binary exponents (GB = 2^30 bytes)The system obviously behaves a bit different from a standard desktop machines, e.g. different areas of RAM are differently fast, depending on the core on which your current process runs, you need to disable individual CPU lines displayed in top etc., but apart from these, mostly everything seems “normal” to me.
this is NUMA, in case anyone 'new to sysadmin' or to architecture is trying to Google this
Still, I agree re. the difference in price. I'd buy too
(I honestly don't know, it's just surprising.)
Databases have inherent locking problems that takes more resources to resolve with more machines, more so in some database systems than others. When scaling you often hit a point where you get down to macro logistics, so imagine a car highway: You can get more throughput by adding more lanes. But what that all cars are going to merge to one lane at some point in the middle of the trip (because of a tunnel, bridge or something: it's impossible to have more than one lane at that point)? Now you won't get any throughput benefits of the multiple lanes after all! Just more latency because you get queues up until the single lane and because of the queues all cars are going slow and need to accelerate on the single lane, so the average speed is low too. You are also having too much resources after the single lane because you can never fill all those lanes. It might be better to have one beefed-up single-lane road all the way that people can go fast on. Basically: Remove locks and you get better overall performance.
Yes, this is on the expense of HA. Yes, the costs of scaling up grows asymptotically faster than the costs of scaling out. So this is definitely a trade-off in some sense.
They just run everything through one live database all the time under normal conditions. Personally, I think this is much simpler to manage than to be regularly spreading out queries over many queries.
If I remember correctly they're using Dell R720's for their SQL Server machines. At current Dell retail pricing 384GB of RAM would be about $5,900.00.
At that price I'd vote for throwing money at raw hardware and reaping the benefits of caching the database in RAM, versus trying to pay programmers to come up with clever techniques to deal with having less RAM.
I don't think addressing that memory is a problem on any x64 architecture. Windows Server 2012 has a 4TB upper limit on RAM, apparently [1], and as far as I can tell there's no reason for this other than product differentiation between different "levels" of the operating system.
1. http://msdn.microsoft.com/en-us/library/windows/desktop/aa36...
Doing this on Amazon r3.8xlarge ($2.8/hr) would cost you $8100+ over 4 months.
A DIY server can make sense in some cases. Eg. you are a very early-stage startup with almost no funding, and you are good at assembling and troubleshooting computers, and are ok with mere warranty on parts as opposed to full vendor support from HP/Oracle/Dell, go for it. That's how many startups started (see Google and their makeshift half-donated half-loaned boxes in 1998).
10tb = roughly $1,000 per month
They're between 5 and 50 times more expensive on bandwidth than the options in dedicated / colo / self hosting.
The more RAM you have the better without exception here.
edit: wrong number
64 bit computers has a memmory limit so large that it is not understandeble.
The ram is still lightning fast compared to other stuff.
No, that's no common for production servers for popular web services, though- they tend to shard across other dimensions because the web services are often CPU or network-bound.
Says in the article:
Windows 2012 is used in New York but are upgrading to 2012 R2 (Oregon is already on it).
This is because you have to effectively (or literally) rebuild the Windows cluster from scratch and we just don't get that level of benefit from the 2012 to 2012 R2 upgrade. There are quite a few improvements we care about: native NVMe, better dynamic quorum, better DSC support, better SMB, and such...but not enough to make the upgrade worth it.
Nick Craver - Stack Exchange Sysadmin & Developer
Trust me, we're bitching about this as are most people and I think changes must be coming there. It doesn't matter how many fancy features you add to the OS if we can't upgrade to it, so they'll have to stop and address that problem.
I have next to zero experience with server administration, but 384GB seems like a lot to me. Is that common for production servers for popular web services? Do you need a customized OS to address that much memory? Seems like you'd really need to beef up the cache hierarchy make 0.38TB of RAM fast.