News for hackers. Stuff that matters.
Re:How Do You Program a Computer with 10 Terabytes of RAM? by dekhn on Wednesday August 05, 2015 @07:21PM (#10013571)
A 2TB RAM machine with lots of high end Xeon chips is only about $120K, probably less if you are a good negotiator.

I've programmed large RAM machines and it's not that hard. In general, it simply let me run programs I couldn't run previously because they allocated too much memory and crashed or swapped/paged too much.

Having a flat memory hierarchy (all RAM has the same cost) makes it dirt simple. NUMA made it significantly harder because you typically had to structure you program's data, threads to schedule on the appropriate cores or processors.

However, I've found over time that unless you absolutely need to hold all your data in RAM, then spending the additional money to get the largest DIMMs and a motherboard with tons of DIMM slots isn't really cost effective. So long as you can partition your problem, that is a preferred solution in nearly all cases. however, increased programmer productivity can often be more cost effective ("just buy more RAM") and I know people who have been upgrading their computers for years and running the exact same algorithms on their in-memory data sets at very high speeds for years.

Depending on your code, some complex problems can occur. For example, the TLB, which speeds virtual/physical address translations, has its own TLB, and if you're screaming all over memory you can blow out the TLB for the TLB. my experience has been the TLB issues go up with larger memory (since you tend to load more data on each node, and access it with sparser patterns).

I haven't looked at DIMM price curves or roadmaps lately, but I assume that 4TB will be $120K in another 3-5 years, then 8TB will be $120K in 10 years.

With all that said, SSDs greatly reduced the issue of needing lots of memory. I've done jobs that would have required too much physical RAM for my budget by configuring linux swap on a fast SSD. It swapped a lot, but the jobs ran :-)

Re:How Do You Program a Computer with 10 Terabytes of RAM? by cmdrfred on Thursday August 06, 2015 @12:10AM (#10014481)
For the programmer who works with far more pedestrian hardware what does someone do with 2TB of ram that you wouldn't implement in lets say a rainbow table on a machine with lesser specs? If you are allowed to say that is.
Re:How Do You Program a Computer with 10 Terabytes of RAM? by dekhn on Thursday August 06, 2015 @11:50AM (#10017297)
Sure (I'm not sure what rainbow tables have to do with this; those are typically for a very specific use. Bloom filters seem to be a better choice; they allow for fixed-memory hashmaps with known false positive rates).

I had a large multidimensional object and I wanted to smooth it.

So, I loaded the whole object in RAM, did an FFT on it, multiplied it by a gaussian, then did an inverse FFT on it.

Another example would be storing very large hashmaps (as a backing store for a sparse multidimensional array).

If you had a very large biosequence database, and you wanted to map next-generation reads against it, you'd load the database into RAM with an mmap call followed by a block read, then subsequent mappings would be very fast.

Effectively, having large amounts of flat access RAM means you don't have to worry about communication (between nodes, or between node and disk).