Re:Why Do Big Irons Still Exist?
by bmh100 on Thursday November 05, 2015 @06:13PM
(#10516813)
I work with a platform that fits exactly into the big iron use case. A typical machine will host millions to billions of rows of transactional data in memory. The round time from a user interaction to fully joined (across several or dozens of tables), aggregated (across several dimensions), and visualized data rendered by the client browser is expected to be less than 2 seconds. Each second of wait is an exponential cost to user experience. Yes, you have a single machine with dozens to hundreds of GB of RAM. You also have a responsive analytics experience, and that value makes it all worthwhile.
[ Parent
| on HN ]
Re:Why Do Big Irons Still Exist?
by GFK_of_xmaspast on Thursday November 05, 2015 @06:42PM
(#10516908)
I've got "dozens of GB of ram" in my desktop, and you can get a 64GB stick on amazon for under $400.
If you have 24 GB of RAM, then you are certainly a rare breed. Curious, what motivated you to install so much?
Re:Why Do Big Irons Still Exist?
by xresonance on Thursday November 05, 2015 @06:50PM
(#10516948)
I've got 64GB, and will double it when I get the other CPU socket populated (bloody expensive Xeons!) This is because I work with giant data sets, in this case, digital video, specifically cinema-quality stuff spit out of a RED camera. I'd do the same with any other large data sets, databases, analytics, etc. RAM is inexpensive compared to the time saved by having data rapidly accessible. Even arguable for web surfing and other light desktop stuff. We can never get more time, but we can make the machines quicker and piles of RAM are a good way to do it.
Is it ECC protected?
Re:Why Do Big Irons Still Exist?
by beachstartup on Thursday November 05, 2015 @08:45PM
(#10517393)
almost certainly, if he's running xeons.
my xeon mac pro is 6 years old and it has it.
i can't imagine running that much ram without ecc. you would kernel panic or spontaneously reboot constantly. not to mention silently corrupt half your work...
Re:Why Do Big Irons Still Exist?
by danieltillett on Friday November 06, 2015 @01:52AM
(#10518137)
Would you be more likely to kernel panic with more non-ecc ram? Won’t a bit flip probability be linked to the size of the critical os code size and that should be the same if you have 2 GB or 64 GB of ram?
If nothing else, the cost savings in no longer having to actually worry about that vector of data corruption are well worth the small delta in cost.
I'm actually rather amazed that at least error /detecting/ ram isn't more common.
I think consumer CPUs don't support it only to maintain market segmentation for server CPUs.
Re:Why Do Big Irons Still Exist?
by danieltillett on Friday November 06, 2015 @02:18AM
(#10518193)
I don’t disagree - it is silent data corruption I really worry about not the odd kernel panic.
Re:Why Do Big Irons Still Exist?
by beachstartup on Friday November 06, 2015 @10:51AM
(#10520060)
> Won’t a bit flip probability be linked to the size of the critical os code size
no, there are a wide range of memory conditions in which a kernel calls panic, and memory doesn't just flip single bits when it errors out, although that's certainly possible.
Re:Why Do Big Irons Still Exist?
by danieltillett on Friday November 06, 2015 @05:34PM
(#10522525)
I thought that single bit flips was all that ecc memory protected against [1].
It is not flip errors per say that causes kernel panics (it is not going to cause any problems if a section of unallocated memory has a bit flip), but the probability of an uncorrected bit flip in a critical memory location. I am certainly willing to be corrected, but if don’t see how increasing the amount of non-critical memory increases the chance of a kernel panic.
not the parent, but for me it's not having to think about what I run concurrently. A browser instance with many tabs, VMs running and editing large files in e.g. Photoshop and you'll easily go over 16 GB. If you regularly compile large projects or develop memory intensive apps you might want more. A few people I know were very happy about new Intel desktop CPUs supporting 64 GB instead of only 32 (and get annoyed every time they try to buy a new laptop and discover that 32 GB still is hard to get)
Re:Why Do Big Irons Still Exist?
by vinkelhake on Thursday November 05, 2015 @06:49PM
(#10516946)
I've got 32GB in my three year old workstation and 16GB in my laptop. Memory is cheap, so why not?
Re:Why Do Big Irons Still Exist?
by GFK_of_xmaspast on Thursday November 05, 2015 @07:16PM
(#10517076)
Firefox on linux was sucking down an incredible amount of memory to the point where I was having to restart it daily, after installing the extra ram it would tend to stabilize around 10GB. (only extension I had running was adblock)
If you were running adblock, you might have hit this bug: https://bugzilla.mozilla.org/show_bug.cgi?id=988266 ("Very high style-sets memory usage with AdBlock Plus").
Adblock is actually pretty memory hungry, if I remember right.
i often find myself using a number of virtual machines at the same time.
also, chrome tabs ain't cheap.
A lot of large companies got onto Oracle a decade or two ago, and now are at a scale where Relational DBs are just not quite enough- but getting off them isn't fast or easy either.
All that's left is a Big Iron, while a new architecture is figured out.
Are you referring to row-store, hard-disk DBs (especially data warehouses) or column-store, in-memory DBs?
Re:Why Do Big Irons Still Exist?
by nickpsecurity on Thursday November 05, 2015 @06:32PM
(#10516873)
Curious, what's your hardware configuration for that?
I mainly work on the smaller side of servers, so you would be looking at Intel Xeon or i7 2.8-3.2 GHz (4-8 cores), 16-32 GB RAM, Windows Server 2008, a 100+ GB SSDs in RAID, 1 TB HDD, with nightly backups. Interestingly, management analytics applications can handle longer downtime than operational systems, so a full system restore from the nightly backup is an acceptable process. Data can recovery can just be performed without downtime, with background processes and zero-downtime database updates.
Re:Why Do Big Irons Still Exist?
by nickpsecurity on Thursday November 05, 2015 @06:48PM
(#10516940)
Interesting. Especially them doing a nightly restore. Many forget to ensure the restore process will work as well as the backup. That company will probably have no problem there. :)
The restore is only as-needed, but full system image backups are made every night.
Re:Why Do Big Irons Still Exist?
by nickpsecurity on Friday November 06, 2015 @04:56AM
(#10518602)
Ah OK. It was too original and extreme to be true. :)