To some extent, that change has already happen.

The RAM is no longer fast: unless cached, it takes around 150 CPU cycles to access the RAM.

The RAM is no longer byte addressable. It’s closer to a block device now, the block size being 16 bytes for dual-channel DDR, 32 bytes for quad channel.

Too bad many computer scientists who write books about those algorithms prefer to view RAM in an old-fashioned way, as fast and byte-addressable.

> the block size being 16 bytes for dual-channel DDR, 32 bytes for quad channel.

For most practical purposes I believe in x86 computers, the block size to consider should be at least a cache line, so 64 bytes.

You need a clean abstraction when describing elementary algorithms at an undergraduate level. A lot of work has been done on cache-oblivious algorithms that respect the memory hierarchy, including from the authors of the classic CLRS book.

Remember the "3M" computer ideal? Mega-Herz, Megabyte, Megapixel, for speed, memory, screen, to make up a new capability of workstation?

https://en.wikipedia.org/wiki/3M_computer

I now think the next target ought to be latency and power defined, in particular with IOT requirements.

Microseconds, Milliwatts, Millions of endpoints.

or, for storage, Microseconds, Millions IOPS, Multi-Parity

or for computer, efficiency, IOT goals:

Milli-watts as a constraint for a benchmark of compute Millions of endpoints / processes or cores on a network topology Microseconds as a measure generically of access to resources, whether memory is sat on another node, or local store, the delay ought to be inside the same order of magnitude as a target.

My purpose in all of that, is, should there - in fact, can there be - consideration of architecture and topology that avoid .. rather can we aim for linear cost in addressing complexity at all, as a goal, or has that been lost already?

I mean "that" and "lost" very vaguely, being non expert, but my question is really should I imagine that there are no effective gains to be had, designing for a IOT style or massively networked future, from the way memory is addressed, or has the complexity that we have, been introduced out of necessity, so will be here to stay, for practical reasons, so the idea of low latency, low power, IOT "grid computing" on a ad hoc basis, is smoke in my pipe?

> It’s closer to a block device now, the block size being 16 bytes for dual-channel DDR, 32 bytes for quad channel.

Don't forget the 8-deep prefetch buffer. The real block size is more like 128 bytes for dual channel.

Differently is right. Every buffer over/underrun can now potentially overwrite your "persisted" data - now we're all writing file/storage/db systems .... ;-)

It'll be wonderful if we actually can get some simplifications in our byzantine architectures, although - I wonder if it doesn't all boil down to our "RAM" (fastest storage) becoming "smarter" - as CPUs manage the three levels of cache - and we "only" loose the "slow disk/ssd" on the far end. CPU cache becomes the new RAM, and cache management microcode becomes our new memory allocators....?

A buffer overrun can't overwrite random memory that isn't mapped into your process. We solved this issue with virtual memory like 35 years ago.

Kind of... We solved this issue like 35 years ago and then broke it again in the last decade. First CPU timing attacks allowed peeking at secret key generation in shared environments. Then we got rowhammer which ignores your mapping. Now we got memory access timing attacks which allow probing kernel memory mapping and bypassing ASLR.

While you can't easily write into other processes anymore, the barrier is more of a suggestion again.

"ignores your mapping" is pretty generous considering rowhammer doesn't work on ECC RAM.

From https://en.m.wikipedia.org/wiki/Row_hammer

> Tests show that simple ECC solutions, providing single-error correction and double-error detection (SECDED) capabilities, are not able to correct or detect all observed disturbance errors because some of them include more than two flipped bits per memory word

But anyway - even if ECC was perfect at preventing the issue, lots of hosts do not use ECC. That includes practically every desktop.

So you're saying there will be a web assembly version in a few years that will claim to be a huge advancement?

Yeah, just like that time microsoft invented lambda functions

Correct, but in practice memory has permissions on a page level and most pages today are 4K. So a page containing a buffer is also likely to contain other data as well. Unless you are going to isolate each buffer by placing it in its own page.

That doesn't make sense to me. So an OS will assign a 4K page of memory to multiple processes, with some granularity that each process "should" only be able to write/read some of it?

So your 1tb linked-list/hashtable that holds all the mail servers email, and is conveniently persisted because ram is the disk, and everything on the heap is magically permanent - isn't at risk?

> Differently is right. Every buffer over/underrun can now potentially overwrite your "persisted" data - now we're all writing file/storage/db systems .... ;-)

Sounds like a great use-case for lmdb :)

Well, many of us are ready today for the new era, writing software differently: not optimizing for memory, throwing out the window ACID databases in favor of quickly hacked together in-memory ones, not knowing any disk-based algorithms and data structures, etc.

/s

Aha, I found what I first came to say in this thread, here is the memory "driver / middleware" origins: http://www.scalemp.com/media-hub-item/scalemp-software-defin...

But ComodoHacker's comment, made me think yet again, why was ACID compliance ever a issue?

I was reliving some nostalgia at the weekend, explaining to a friend how I got excited Microsoft Transaction Service came bundled with NT.

That was my "free" ACID transactions.

It as cross platform, then, too. (or at least multi arch, and advertised to play nice with things like IBM CICS, which may have been much of the point of the bundling, to win deals, even in check box tallies)

MTS is one of the few OS level dependencies SQLServer has. But i think the cross platform origin I recall, was born out when SQLServer/Linux appeared.

I have all this time been confused.

I get it, if you don't need ACID, and have other design reasons, go ahead.

But, well I just always saw it as a problem solved, or plug and play solvable, thanks to MTS. Getting a book on MTS is hat convinced me NT was serious and our business should take heed. If you're a small shop, have margins, can work with fat servers - or as we did, fundamentally scale through transaction routing in the first place - NT (plus the variations of services for Unix now for Linux) can be a happy place.

Incidentally, I believe the middleware - driver that Intel is shipping with Optane, is the work of ScameMP.

This is their Flash Expansion description: http://www.scalemp.com/products/flx/

we've used their product very happily - it may be a good fit especially if you are staging oversize db's you intend to shard, but need them up behind a connection while you test, which was our need.

N.B. Scale MP has a free tier which may also fit your needs.

Not meaning to shill, but I never know why I don't hear of them more, our experience was absolutely satisfying.

edit: typos; "free"/free edit: removed "very" from "may be a very good fit" about ScaleMP - felt so for us, but can't say why anything that works to meet specific need is qualitatively better when doing the job is a binary y/n..

and to add, now if products like Optane eliminate the performance cost discarding ACID transaction compliance has been justified by, then will that not upset a few conceptual applecarts? I mean, I think some very loose and fast argument has been given, surrounding non ACID compliance and data and databases generally, over recent years, and certainly the performance cost trade off arguments has appeared to me to mask logic elsewhere that needs attention.

Those old enough will remember the AS/400 (now called iSeries) computers map all storage to a single address space. You had no disk - you had just an address space that encompassed everything and an OS that dealt with that.

Just in case anyone else is wondering:

https://en.wikipedia.org/wiki/Single-level_store

I must admit that until looking into it more closely i figured it was no different from the memory map of a micro. Silly me.

What were the pros and cons of that design?

I never programmed for one myself, but I imagine that it's a very alien world if you see everything as a single vast address space of persistent objects. I imagine layers of software would make it less strange in order to facilitate porting programs to the platform - I remember IBM ported Node.js to it, so, there must be a way to see it as a more traditional machine.

Bugs become a lot more persistent, for one. "Reset it and see if it goes away" stops working nearly as well.

The end is near! Only Haskell and purely functional programming (persistent shared data structures) will save us from persistent bugs.

Considering how many odd state issues I see these days (more often in Angular 1.x apps than anything else), don't underestimate the potential for problems there.

Keep in mind that Smalltalk also persisted everything in what I think was called images, and it managed fine.

Already learned the joy of waiting for a Surface to drain, to force a reboot. Forbidding shutdown/reboot without authentication on the network was the proximate cause of the wait. But not being able to remove power, from a device that was not trusted, and could be doing anything in theory, was a sobering thought.

Should devices like tablets, with enclosed batteries, be required to have physical circuit breakers for power, as a security measure, if compromised?

We realized that we had to have a planned action to block MACs from the network, in response to any device having questionable integrity.

Even a small Faraday cage was considered - WiFi not the only radio on portables: laptops without removable batteries hop across LTE via VPN... so that was another policy to set a script to shut down.

Persistence, in this case of processes - or just not being able to remove a battery - is a threat that with good reason shocked us, because it is so ubiquitous. I believe the moment a unscheduled reboot or shutdown occurs, at the very least all LAN/WAN access needs to be automatically cut.

edit:typo

Won't OS just have a service to 0 out all the RAM, definitely on shutdown for security reasons, with a performance hit in time and write cycles on your sdd. Same mechanism at boot?

That assumes that the OS didn't crash. It also assumes that the data isn't valuable, in which case why not just use faster, cheaper ram?

There are workarounds, like adding a layer of faster, cheaper ram, but then this starts to look like a big perf improvement in a rather traditional system.

And on startup, if anyone could have physical access to your hardware.

>What happens if the distinction goes away? What if all data is persistent?

A while ago (a few years?) there was some buzz about a new kind of machine from HP (maybe based on memristors? which were also in the news around then, IIRC), that was supposed to do away with that distinction. They called it The Machine or something like that (:-). I did not follow it at the time, after the initial read, so don't know what happened to it.

The results of the research being made into a product now, managed by a new foundation named Gen Z.

"BUD17-503 The HPE Machine and Gen Z" at Linaro Connect 2017

https://www.youtube.com/watch?v=1BVtChDQVyQ

Thanks, will check that out.

Keith Packard also did a number of talks at linux.conf.au over the years about various bits and pieces of the machine

Interesting, thanks. I recognized his name as one who worked [1] on the X Window system, from a book in an X/Xlib/Motif course [2] I attended much earlier.

[1] https://en.wikipedia.org/wiki/Keith_Packard

Googled his name and found this:

A look at The Machine [LWN.net]:

https://lwn.net/Articles/655437/

[2] On a side note, Motif was pretty powerful. I remember a colleague of mine who was also in that course, creating a rudimentary app like MS Paint, in the class, in just half an hour or so - as the instructor was teaching, in fact. (Maybe he had studied it some before, of course.) As he demoed it to us, he went "Foo!" :)

CMU has been conducting research in this area for a few years now.

http://www.pdl.cmu.edu/NVM/index.shtml

I don't know. I feel like instead of reducing memory tiers Optane will only add another layer in the stack. It is still 10x slower than DRAM, which sounds quite significant. Besides, I'm not sure if persistence is really the most important aspect of storage anyways, especially with modern-day always-on clustered systems.

Here here. 12 factor has taken us out of the age where this should matter. Don't drag us back and tell me disk is as good as memory now.

Many of us have always made the distinction between cache lines and system memory. While this will probably be a welcome improvement it's not like all of your latency problems magically disappear.

I've thought about this, and I don't think that making all data persistent is necessarily a good idea. Programs tend to crash and corrupt their volatile memory. If you want some data to be reliable and permanent, you have to treat it as such.

The main obvious advantage of using non-volatile RAM across the board is that it won't consume power all the time, like DRAM.

"Persist all data" shouldn't be taken as a literal notion. It's better to think about decisions of persistence eventually moving from strongly hardware influenced to purely software- and problem- defined. Just as we've developed abstractions and tools at the persistence ("filesystem") layer now, like snapshotting, similar tools will evolve to take advantage of the new hardware constraints.

I'm not sure that the hardware has been that much of a constraint lately. Evolution, not revolution.

I mean how long ago did we have 32 mb of ram or less? Now we have 32 GB of ram in personal computers. This is more of the same progression, right?

And roughly the same period of time further back (give or take), we had 32kB of RAM.

I think the title may be a bit far fetched. They are talking about 60-100 microseconds latency. DDR3 is usually 9 nanoseconds latency.

L2 cache is several nanoseconds. DDR is much slower.

This page says 100ns: https://gist.github.com/jboner/2841832

This one says 60ns: http://stackoverflow.com/q/4087280/126995

Got these 9ns from the wikipedia [1]. But 60-100ns would still be 3 orders of magnitude faster than this SSD.

I think the difference comes from the latency of operation from the CPU point of view vs actual memory/drive latency.In the article they seem to be talking about the drive latency.

1. https://en.wikipedia.org/wiki/DDR3_SDRAM#Latencies

That text is talking about the CAS latency, not full read/write latency. CAS latency is the latency for reading from an open row. Rows are small enough that the only way you can persistently hit an open row is if you are either reading sequentially, (in which case the CPU prefetch logic makes the memory latency go away completely) or if you work on such a small working set that you actually hit the cache all the time anyway.

On DDR3, the full latency of a miss to ram is, in a low-bandwidth scenario:

      latency of the full cache system, typically expressed as the L3 latency
    + latency of opening a row (Trcd)
    + latency for reading from an open row (Tcas)
    + latency of passing data from memory controller to L1 cache

In a situation with some but not anywhere near full load on the memory subsystem:

      caches
    + latency of closing a row (Trp)
    + latency of opening a row (Trcd)
    + latency of reading from an open row (Tcas)
    + latency of passing data to L1

and when doing full tilt random access:

      caches
    + time remaining waiting out the minimum allowed row active time of the previous memory access (Tras)
    + latency of closing a row (Trp)
    + latency of opening a row (Trcd)
    + latency of reading from an open row (Tcas)
    + latency of passing data to L1

The total latency is typically between 50 to 150 ns depending on the CPU and RAM in use.

Or maybe 3x9 if you are doing wild random accesses in big area, or even worse in huge area if your TLB and then caches start to not hold enough and you effectively need to do multiple accesses for each single ones, but you are right: DRAM is faster. However, caching Optane with real DRAM seems a really good idea (or maybe using Optane as swap could be useful too)

Not shockingly, we've already moved beyond caring about local ram for many problems. Any system that is running distributed should be taking into account how long it takes to get data from across the network.

Similarly, the difficulty in large data problems is often more of getting the data in the first place. This is often about coordinating the data creation of many machines and systems.

So, for most of us, this shift will not be huge. Because it is mostly irrelevant. Even systems that do have this much data often benefit from algorithms that don't rely on directly accessing and processing all of it in one shot. If only for resilience. (That is, if a system failing doesn't mean reprocessing all of the data, restarting and recovery is often faster and easier to deal with.)

"As above, so below" comes to mind.

Sometimes it seems that the diff between a CPU and a cluster is the suffix put on the latency times.

I thought I read somewhere that the fastest networking technology actually gets close to a couple hundred instructions window of processing.

http://netoptimizer.blogspot.com/2014/05/the-calculations-10...

Hm, 67 ns per packet.

So even this local storage would be the drag in many scenarios...

The mapping layer to treat Optane as RAM, comes from ScaleMP. They have a free tier product. For heavily read intensive databases, cheap read optimize SSD can work pretty well with that. Now I don't have to also buy a product license with Optane, to do that, the price for the Optane drive looks much better to me.

Not sure drive is quite the right word, any more. Has a better nomenclature settled yet?

Computer Science is not Computer Engineering. If anything, much of CS is pure theory, relying instead on abstraction of underlying computation device, called Turing Machine.

RAM technology hasn't stood still. There's multiple levels of cache, wider channels, etc. There will be different levels of performance for a while now.

>My guess is that we'll start writing software very differently. It's hard for me to predict how, though.

This is huge deal. SSD's started it. It always seemed wrong for me to address memory through a disk interface.

The main issue is that the re-design of software for permanent non-volatile storage is huge. It goes back decades, and there are many design decisions that need rethinking.

I may be wrong, but I am guessing that database software could be the first class of software to really take advantage of this shift.

There has been some outliers though. I seem to recall Palm used something called "run in place" on their early generations. But then perhaps storage on those devices were battety backed RAM.

Well that should obsolete ORMs! There's no impedance mismatch when there's no RAM/Disk. We'd need more Disk (DB) components running for Persistent-RAM (indexing, transactions.)

You'd be surprised the number of devs I know that have a hard time thinking in terms of memory-mapping... the other issue is upgrading structures to add/remove fields will become much more difficult that way.

> Well that should obsolete ORMs!

Why? ORMs are essentially systems to convert data structures. Adding a new layer of memory does nothing to address the problem.

wow, and here I thought that SSDs were disks made of RAM