> The Optane Memory products are squeezing into a relatively small niche for limited budgets that require a lot of storage and want the benefit of solid state performance without paying the full price of a boot SSD. Intel notes that Optane Memory caching can be used in front of hybrid drives and SATA SSDs, but the performance benefit will be smaller and these configurations are not expected to be common or cost effective.
It doesn't sound that fast. Looks to be "slower" (throughput-wise) then modern NVMe drives (i.e. Evo 960 and obviously smaller) and clearly trying to bridge a price gap. Is the upside that it can act as an additional cache level without OS support? I remember the first time I used Intel Rapid Storage Tech (aka RST) ... it was also the last time as it's was convoluted with minimal (or no?) upside on Linux.
As an "enthusiast" I'm assuming that my Linux desktop with plenty of RAM + NVMe backed lvmcache will perform better.
I also assume that my laptop with reasonable RAM (because DDR (self)refresh sucks enough power to annoy me) + high performance NVMe drive also won't benefit.
Re:Optane Memory – A bridge between DRAM and storage
by 2bluesc on Monday March 27, 2017 @01:26PM
(#13969790)
Interesting comment from Anandtech [0]
> Keep in mind that consumer NVMe SSDs that boast throughput of 2GB/s or more generally do not reach their peak at low queue depth. Optane is supposed to be able to drive 1200MB/s read throughput at low queue depth (not sure why they listed QD4), so there is potential for some performance improvement here. Most consumer workloads never get out of low queue depth territory, so this could have some small real world benefit. Write throughput, however, is critically low.
> More importantly, these Optane drive are gear more towards lowering latency than transferring large files. Where HDDs access the data on the order of 10s of mS and SSDs access data on the order of 1mS (give or take), Optane should be able to access data on the order of 1s - 10s of uS. Where Optane will be useful is high numbers of small file accesses (DLLs, library files, etc.).
> That all said, I'd just as soon leave all the extra complications, compatibility issues, and inconsistencies on the table and get that 2 GB/s sdd that you mentioned until Intel figures out how to make these more compatible and easier to use without requiring a "golden setup". I don't want to buy a new W10, Kaby Lake, 200 series based system just to use one of these. My current W7/W10/Ubuntu, Skylake, 100 series system should work just fine for a good while yet.
Re:Optane Memory – A bridge between DRAM and storage
by ascheidecker on Monday March 27, 2017 @01:47PM
(#13969985)
Another factor is that these consumer Optane products only have 1 or 2 chips to multiplex operations across. Their enterprise product, the SSD DC P4800X uses 28 of the same 16GB chips and can read or write at ~2GB/s which is competitive with NVMe SSDs.
Re:Optane Memory – A bridge between DRAM and storage
by runeks on Monday March 27, 2017 @05:11PM
(#13971805)
As far as I know, SSDs are closer to 0.1 ms latency (~10,000 random read IOPS at zero queue depth). That's still an improvement of more than an order of magnitude, though (0.009 ms according to the article).
Re:Optane Memory – A bridge between DRAM and storage
by sliken on Tuesday March 28, 2017 @03:18AM
(#13974711)
Dunno, I find 1.2-1.6GB/sec on my samsung s950 all the time. The benchmarks I've seen do not show any noticeable difference between SATA and NVMe. Sure geeks like the idea of a lower head protocol, but truth is the protocol is not the bottlneck.
Seems like Optane is a solution looking for a problem to fix. Normal M2 SATA's do fine:
# dd if=/dev/sdb of=/dev/null bs=131072
1953586+1 records in
1953586+1 records out
256060514304 bytes (256 GB, 238 GiB) copied, 159.18 s, 1.6 GB/s
Re:Optane Memory – A bridge between DRAM and storage
by wtallis on Wednesday March 29, 2017 @01:46AM
(#13983756)
You're not getting 1.6 GB/s out of a SATA drive that's communicating over a 600MB/s link. I think you may be confused about which products are using SATA vs PCIe.
Re:Optane Memory – A bridge between DRAM and storage
by slizard on Monday March 27, 2017 @01:48PM
(#13970000)
Partly it is just marketing fluff, Intel trying to save their face and hoping that most will not remember the promises of revolutionary product while what they released is barely evolutionary.
Can't find a better article right now than this SEmiaccurate rant [1] which is quite inflammatory, but does not push falsehoods AFAICT. But Intel has essentially walked back from the 100-1000x improvements it initially claimed to something that's barely better than NVMe in terms of performance and robustness seems far from initial claims too.
Re:Optane Memory – A bridge between DRAM and storage
by tw04 on Monday March 27, 2017 @03:37PM
(#13970973)
No, they really weren't. Early CHEAP SSDs from the likes of OCZ were horrible. I STILL have the first Intel SSD I bought - 80GB. It was awesome then and it's still awesome now. It's probably half the speed of modern SSDs, but even then it was several orders of magnitude better than any spinning rust that came before it.
The first gen SLC - from the likes of STEC were as fast if not faster than anything we have now, they were just ridiculously overpriced.
Re:Optane Memory – A bridge between DRAM and storage
by runeks on Monday March 27, 2017 @05:21PM
(#13971877)
The limitation back then was the controller. Intel figured out how to do it properly before anyone else. I still have an 80 GB X25-M G2 as well (read latency 65 µs [1]).
Re:Optane Memory – A bridge between DRAM and storage
by fulafel on Tuesday March 28, 2017 @01:00AM
(#13974210)
The first SSDs were DRAM based and pretty fast for the time, see eg. https://en.wikipedia.org/wiki/Cray_Y-MP. (SRAM, now used in CPU caches, was used for the whole main memory on that machine)
Re:Optane Memory – A bridge between DRAM and storage
by petra on Monday March 27, 2017 @02:11PM
(#13970204)
I understand why Intel claimed 100-1000x improvement: it's great for the stock. But what happens now if it's not true, will the stock loose all that gain ? or some of that gain will stay ?
Re:Optane Memory – A bridge between DRAM and storage
by slizard on Monday March 27, 2017 @02:54PM
(#13970583)
I don't think there's anything illegal in ove-promising and severely under-delivering. Also, given the long delay of the 1st gen product release Intel's massive PR and marketing machinery had plenty of time to rewire many if not most people's expectations. Doesn't have the endurance and the orders of magnitude performance advantage promised? Just focus QD1 performance, feed 3-5-10x instead to the media for long enough, over-provision to be able to deliver an OK consumer product and keep everyone's hopes up that revolution will come "soon".
People have short memory (even tech journalists), especially when there is not enough competition.
Re:Optane Memory – A bridge between DRAM and storage
by dragontamer on Monday March 27, 2017 @03:21PM
(#13970805)
Its pretty crap for the consumer market. Your typical MLC SSD (or even TLC SSD) over SATA is more than sufficient for the consumer market, at prices that actually will be used (although even a TLC SSD seems to be considered "high end" by my family members...)
---------
The Optane memory seems to have a niche in replacing DRAM servers. So instead of running Redis, MySQL, or Postgres on DRAM, you can run it on Optane instead.
Optane is cheaper than DDR4 but almost as fast (latency wise). Latency is a big deal when it comes to database and/or key-value pair workloads.
Re:Optane Memory – A bridge between DRAM and storage
by usrusr on Monday March 27, 2017 @07:07PM
(#13972623)
One presentation was comparing an 8GB+HDD system to a 4GB+Optane+HDD system, showing off how the low RAM hybrid was much faster at selected tasks (carefully selected tasks, I presume). This is probably the market, another indication is the surprisingly low price.
The price difference between 4 GB and 8 GB of RAM cannot be enough to compensate for the Optane module, but if the Optane combination uses less battery than the bigger RAM and if enough benchmarks and/or magazine performance comparison use cases coincide with tasks where the hybrid shines, then some low end manufacturers might successfully sell the RAM downgrade as an upgrade option.
Re:Optane Memory – A bridge between DRAM and storage
by wtallis on Monday March 27, 2017 @08:28PM
(#13972978)
> "but if the Optane combination uses less battery than the bigger RAM"
With idle power specified as 0.9-1.2 W, I'm not expecting the Optane-based solutions to come out ahead on any battery life tests. PCIe power management is a horrific mess.
Re:Optane Memory – A bridge between DRAM and storage
by lallysingh on Monday March 27, 2017 @04:13PM
(#13971293)
This. A RAM based cache sounds substantially more performant. I think this may be for clients that can't do much more than swap hardware to make their setup faster. Either it's a lack of control over their OS setup, or a desire to fine-tune caching for a specific device, or a lack of ability to change existing settings.
Re:Optane Memory – A bridge between DRAM and storage
by ChuckMcM on Monday March 27, 2017 @02:58PM
(#13970621)
The 'win' in theory for this stuff is that it is much faster than SSDs at low queue depths and it is significantly more write tolerant.
An interesting interesting system design might be a 2GB main memory with a 32GB swap partition on this device. That would be interesting because in idle mode it uses less power than an equivalent amount of DDR memory. So you could potentially build a laptop with good performance on larger memory tasks but longer standby battery life. It would also be interesting paired with an equivalent amount of DRAM where it mirrors main memory so you could have 'instant' off and 'instant' on type booting and shutdown.
At larger pool sizes this becomes interesting as a non-volatile index for NoSQL and other databases. Intel's claim has been in the past that its density can exceed SSDs by a healthy margin.Not seeing that in these products but time will tell.
I've been watching them struggle to productize this tech for about 2 years now[1]. Now to see if they can get it to live up to the hype.
Re:Optane Memory – A bridge between DRAM and storage
by mtgx on Monday March 27, 2017 @04:27PM
(#13971447)
> Intel's claim has been in the past that its density can exceed SSDs by a healthy margin
Seems doubtful they would reach that goal until SSDs hit some kind of hard plateau (kind of like HDDs did).
Toshiba plans to make a 100TB SSD over the next couple of years, and I believe Samsung has already announced a 16TB SSD. I doubt 3D Xpoint can catch-up with those in the given time frame.
Re:Optane Memory – A bridge between DRAM and storage
by wtallis on Monday March 27, 2017 @08:38PM
(#13973014)
NAND flash memory already hit the wall for horizontal shrinks, and some products plowed past that limit and into the regime where day to day use was invoking the second level error correction methods to compensate for data degradation.
3D NAND turned the clock back on horizontal scaling and cell sizes by several years, and now the only scaling opportunity is to increase layer count. That's not going so well. Samsung started with 24 layers but didn't ship retail products until they hit 32 layers. Their 48 layers was about a year late and caused several products to be scrapped entirely. Toshiba's on their third generation 3D NAND and it's only just starting to show up in the market. SK Hynix is doing worse than Toshiba. Intel and Micron started off reasonably well with their 32-layer and are working on ramping up 64-layer.
By the end of this year, all four manufacturers should be shipping 64-layer (72 for SK Hynix) 3D NAND in volume, or else it's time to admit that 3D NAND just doesn't scale up as well as we hoped it would.
There's some reason to believe 3D XPoint memory might be able to avoid the worst vertical scaling problems that are holding back 3D NAND flash. Each pair of 3D XPoint layers can be pretty much isolated from the rest of the layers except at the edges, whereas 3D NAND is building columns of bits that run through all 64+ layers and have aspect ratios that are quite high.
Re:Optane Memory – A bridge between DRAM and storage
by markvdb on Tuesday March 28, 2017 @05:17AM
(#13975175)
I can see some potential in things like RERAM to scale beyond 3D NAND and bridge the SSD/RAM gap. I like the technological elegance of non-filamentary RERAM [2] in particular.
Re:Optane Memory – A bridge between DRAM and storage
by freeone3000 on Monday March 27, 2017 @01:14PM
(#13969634)
It looks to be an accelerator cache for HDDs, but slower than an SSD, with fairly low capacity. And it requires a 7th gen intel processor, so this isn't a performance boost for older PCs either. Exactly who is this aimed at?
Re:Optane Memory – A bridge between DRAM and storage
by valarauca1 on Monday March 27, 2017 @01:29PM
(#13969824)
Nobody.
You lose SSD transfer speed, while gain _some_ hardware response latency.
NVMe response latency (on hardware side, not OS) is around 14-15micro-seconds on reads, and tops out at 200micro-seconds on writes [1]. Which is nearly 2-3 orders of magnitude below spinning rust. While Optane can halve this to 7micro-seconds on reads. The price/pref vs established NVMe tech is non-existent.
When I can spend $X/GB to get a 2000% performance bump, its hard to justify spending $X*4/GB to get 2050% performance bump. Most applications/data centers have yet to transfer to NVMe. Offering marginal gains a sizable markup is a risky strategy.
Furthermore Optane is nearly 2 orders of magnitude lower than RAM in latency, and 5 orders of magnitude lower transfer. So the idea of "non-volatile RAM" has yet to arrive.
---
Also "Transparent to the OS" should strike fear in your heart. The OS's job is to manage hardware, memory, and caches. Why drop this?
Re:Optane Memory – A bridge between DRAM and storage
by angry_octet on Monday March 27, 2017 @06:37PM
(#13972456)
The latency of a regular SATA SSD is more than good enough if you want a log of writes to spinning rust. The increased write endurance might be appealing for something like a ZFS intents log (ZIL).
Too small to provide much of an cache, when you can have 256GB of SSD for the same price.
It is a pity that the marketing docs don't address how this impacts on write consistency. Instant write sync is the primary advantage of hardware RAID, but still, how much better is this than a normal SSD?
Re:Optane Memory – A bridge between DRAM and storage
by temuze on Monday March 27, 2017 @01:19PM
(#13969705)
Looks faster than an SSD to me. From the FAQ:
"Q4: Will Intel Optane memory also accelerate an SSD? If so how much?
A: Yes. Intel Optane memory can be used to accelerate any type of SATA-based storage media, including SATA SSDs, However, the performance benefit of adding Intel Optane memory will be greater on slower storage devices like an HDD versus a relatively faster storage device like a SATA-SSD. Additional performance data is forthcoming"
Re:Optane Memory – A bridge between DRAM and storage
by timothycrosley on Monday March 27, 2017 @01:41PM
(#13969936)
It's faster than SATA SSD but slower than most NVMe M.2 SSDs: http://www.anandtech.com/show/11227/intel-launches-optane-me.... Because of when this was launched, and the fact it only works with newer processors that also support NVMe, by the time it is at all common place, so would M.2 SSDs, making it a weird concept. It's essentially the most expensive per GB NVMe drive, with worse performance, and some software to automate caching.
Re:Optane Memory – A bridge between DRAM and storage
by fgonzag on Monday March 27, 2017 @03:32PM
(#13970906)
Except for the fact that it hits max throughput at way lower queue depths, is bit addressable, and it destroys mixed workloads since you can read while writing. I don't think it'll be revolutionary, but it will be quite nice for caching, databases, or even as a write cache using something like ZFS's zil, then flushing to NVMe SSDs
Re:Optane Memory – A bridge between DRAM and storage
by jandrese on Monday March 27, 2017 @05:16PM
(#13971845)
I don't know, that sounds like a pretty small niche at the end of the day. If it describes you workload I would jump on it ASAP because a year from now it will probably be discontinued.
Re:Optane Memory – A bridge between DRAM and storage
by wtallis on Wednesday March 29, 2017 @02:00AM
(#13983791)
Just FYI, there's no such thing as "processors that also support NVMe".
Intel's Windows-only caching software for the Optane Memory product will refuse to work on any but the newest Intel platforms, but any other caching software will work fine on any platform that supports PCIe. If it turns out that small Optane caches are worth what Intel's charging for the hardware, then the existing third-party caching solutions will get tuned to make full use of Optane.
Re:Optane Memory – A bridge between DRAM and storage
by 1001101 on Monday March 27, 2017 @01:28PM
(#13969814)
My interpretation is that it is a fancy cache using a new type of memory, XPoint, a non-volatile phase change memory, with greater endurance (and cost) than NAND sitting on the M.2 bus, combined with OS drivers to assist with the mapping.
The idea is M.2 speed > SATA, but the dynamic M.2 caching isn't native to the OS, so drivers are required to intermediate.
Re:Optane Memory – A bridge between DRAM and storage
by dv_dt on Monday March 27, 2017 @01:41PM
(#13969930)
M.2 is a hardware connector. It can carry various flavors of SATA or NVMe over PCIe lanes. Generally M.2 drives will be at least equal to a SATA drive on a traditional SATA connector, with the potential to be much faster. However, even some of the m.2/NVMe drives which are much faster at the bus level seem to be bottlenecked by some design assumptions (both inside the drive and above in the various OS layers). I think in the medium to longer term M.2/NVMe will be much faster.
Re:Optane Memory – A bridge between DRAM and storage
by sliken on Tuesday March 28, 2017 @03:23AM
(#13974726)
Just buy a nicer M2 and you get plenty of performance without special drivers:
# dd if=/dev/sdb of=/dev/null bs=131072
1953586+1 records in
1953586+1 records out
256060514304 bytes (256 GB, 238 GiB) copied, 159.18 s, 1.6 GB/s
Re:Optane Memory – A bridge between DRAM and storage
by dv_dt on Tuesday March 28, 2017 @10:13AM
(#13976701)
If you don't test read with some writing, you might be misled by performance of an SSD. 100% read or 100% write are very different from 50/50 or even 70/30 r/w. The mixed bandwidth typically drops by a lot. If you haven't pre-written some data, that dd test might not even be touching the flash. Just reading internal metadata...
Re:Optane Memory – A bridge between DRAM and storage
by codebook on Monday March 27, 2017 @01:34PM
(#13969871)
The only thing could get from Optane is QD1 performance. It is impressive (5x faster than current gen NVMe SSD). But everything else is just soso compared to NVMe SSD.
Re:Optane Memory – A bridge between DRAM and storage
by smlacy on Monday March 27, 2017 @01:10PM
(#13969589)
Quite a bit of marketing-speak here on the official site, without a lot of technical details.
I know the Optane SSD drives are fast -- is this just device-side driver software that allows you to virtually expand available "RAM" and mmap it to the SSD?
Why just i7 CPUs? Is there a hardware component?
"Just download and install the driver..." (Linux?)
Re:Optane Memory – A bridge between DRAM and storage
by wtallis on Monday March 27, 2017 @08:45PM
(#13973051)
These consumer Optane Memory drives rely on a chipset hardware feature (that was introduced in the last generation) to work with Intel's caching strategy for Windows, which is complicated and ugly partly because it has to work around Windows limitations and partly because it has to enforce their product segmentation strategy. On Linux, it's just a NVMe device that can be used with any pre-existing caching software solution.
On the enterprise side, Intel is offering the Memory Drive Technology software that is a minimal hypervisor to present a pool of memory equal to about DRAM+85% of Optane, with the actual allocation managed by the hypervisor and opaque to the guest OS. The hypervisor can run on several generations of Xeon processors.
Re:Optane Memory – A bridge between DRAM and storage
by bbulkow on Monday March 27, 2017 @03:41PM
(#13971006)
Some of us have technical details, and put them out during the press event. The fact that Intel is staying high level is annoying, I admit, but they want "the community" to speak up, when you start using these drives.
I have tested on these devices, as a datastore for databases, and it has benefits. If you're writing a lot, and/or are very latency sensitive.
Re:Optane Memory – A bridge between DRAM and storage
by AdamJacobMuller on Monday March 27, 2017 @01:13PM
(#13969622)
Seems like its some i3/i5/i7, only recent ones? Must be some hardware component to it that makes the SSD look like memory to the system, perhaps as another NUMA zone (could you have a huge NUMA zone with no CPU attached?)
Re:Optane Memory – A bridge between DRAM and storage
by misja111 on Monday March 27, 2017 @01:17PM
(#13969675)
What I understand of it, after skipping through all the marketing speak, is that the Optane Memory functions a cache for your SATA drive. The Optane cache is just a Optane SSD which uses some new and apparently faster 3D Nand technology.
And about Linux; somewhere in the PDF with the technical details it states the requirement that your OS has to be 64 bit Windows ..
Re:Optane Memory – A bridge between DRAM and storage
by wtallis on Monday March 27, 2017 @08:50PM
(#13973072)
> "The Optane cache is just a Optane SSD which uses some new and apparently faster 3D Nand technology."
Optane products use 3D XPoint memory, which is not 3D NAND flash memory or any other kind of flash memory. It's probably a phase change memory, but Intel won't say so we have to wait for Chipworks to put it under an electron microscope.
The Optane Memory product for consumers is entirely centered around the use case of running Intel's SSD caching software for Windows, which requires a Skylake or newer chipset plus boot firmware that will only actually be available for Kaby Lake systems. The hardware feature they're relying on is a disgusting hack that Linux users want to stay far away from (and can, since there are generic caching solutions for Linux that work atop any block device).
Re:Optane Memory – A bridge between DRAM and storage
by amluto on Monday March 27, 2017 @03:05PM
(#13970688)
I think you can (pending driver support?) plug one of these in and memory map it. On Linux, this means you'd see it as a "DAX" filesystem, and if you mmap(2) a file, you get a bona fide direct mapping of the hardware into your address space. This is very cool and would enable some interesting database-ish use cases.
Re:Optane Memory – A bridge between DRAM and storage
by wtallis on Monday March 27, 2017 @08:53PM
(#13973086)
No DAX yet. Both this consumer Optane Memory product and the enterprise Optane SSD announced last week are standard NVMe devices. For the enterprise market, Intel is providing mmap capability with a hypervisor-based software solution.
Re:Optane Memory – A bridge between DRAM and storage
by amluto on Monday March 27, 2017 @06:21PM
(#13972353)
That doesn't mean it can't expose a big physical address window (BAR) to the host. There's some documented precedent: the "controller memory block". As currently specced, though, the CMB is volatile.
Re:Optane Memory – A bridge between DRAM and storage
by tetraodonpuffer on Monday March 27, 2017 @02:17PM
(#13970255)
it seems this would be a great use case for DAWs and sample libraries in general, many many small files to be accessed with extremely low latency: this could make it possible to run large orchestral templates fully purged at all times, which would be a great use case when your template has hundreds of GB of samples but you only use a few of them at a time.
This is already kind of doable with SSD, but if optane has an order of magnitude less access time it would of course make it even better for this use case.
Re:Optane Memory – A bridge between DRAM and storage
by readams on Monday March 27, 2017 @01:24PM
(#13969767)
They're a long way from realizing the full potential of the technology. Right now they're hugely limited by the interconnect. Also, if they reach some of the goals they've been talking about, it might need differently-designed operating systems to really work.
Re:Optane Memory – A bridge between DRAM and storage
by snuxoll on Monday March 27, 2017 @01:36PM
(#13969894)
I mean, Single-Level-Storage is hardly a new concept - I don't think we need entirely new operating systems to gain benefits, but for more applications and enabling technologies to be designed around it.
LMDB, for example, would be a great fit with Optane already. Really, anything that uses mmap() is.
Re:Optane Memory – A bridge between DRAM and storage
by blacksmith_tb on Monday March 27, 2017 @01:23PM
(#13969745)
It seems like that could be pretty useful in a laptop, if your RAM was nonvolatile, wake from sleep would be truly instantaneous, you'd think (and it would survive your battery dying).
Re:Optane Memory – A bridge between DRAM and storage
by ksec on Tuesday March 28, 2017 @11:34AM
(#13977467)
I think this is an Easy win for DIMM in servers where you could fit a much larger capacity then DRAM. And this is only a first gen product.
But for consumers? I am not sure if they can feel the difference between a decent SSD drive and Optane Memory.
It fits the role of Page File. So you could always have 2x - 3x the Optane Memory as Page Cache. System with this would means less Memory required, and saves battery. While the controller for SSD could be less complicated and focus primarily on Seq/Read Write. SSD could also be DRAM-less, save cost and battery.
But even then, $44 for 16GB is just hard to convince most consumers.
Re:Optane Memory – A bridge between DRAM and storage
by ak217 on Monday March 27, 2017 @01:59PM
(#13970111)
If I'm reading this right, this device is slower than a Samsung 960 Evo across the board (http://www.anandtech.com/show/10833/the-samsung-960-evo-1tb-...). It seems that when Intel started developing Optane, they didn't anticipate how much faster NVMe PCIe SSDs would become. Couple that with significant clock frequency headroom for DDR4, and this product really doesn't look competitive.
Re:Optane Memory – A bridge between DRAM and storage
by olavgg on Monday March 27, 2017 @02:11PM
(#13970205)
You're reading the wrong numbers. Try benchmarking with PostgreSQL's pg_test_fsync or with fio with the following options: --direct=1 --sync=1 --rw=write --bs=4k
Good random-write performance with 4k/8k blocks at QD1 is what matter for SSD's, not if they can write 2500MB/s to a volatile cache.
Re:Optane Memory – A bridge between DRAM and storage
by Quequau on Monday March 27, 2017 @01:21PM
(#13969718)
I think a lot of this next generation solid state storage level memory is going to wind up being ill suited for the dominant interconnect standards (e.g. DDR, SATA, or PCIe). I also really wonder if it isn't going to take a really long time for the relevant parts of the existing software stack to see serious development to effectively exploit this stuff.
Re:Optane Memory – A bridge between DRAM and storage
by whalesalad on Monday March 27, 2017 @02:54PM
(#13970588)
Before the page could even finish rendering I had a pop-up to answer a survey. What is this called and why do websites continue to do it? Does this not massively destroy your conversion or retention rates? http://i.imgur.com/90VHYnU.png
Re:Optane Memory – A bridge between DRAM and storage
by patrickg_zill on Monday March 27, 2017 @07:53PM
(#13972848)
2GBytes/second as RAM is as good as DDR1 memory.
So take a CPU with more memory channels on it, run at least 1 Optane per channel, run hundreds of VMs on it, everything runs from this slower RAM, but it is in memory and you only touch the SSDs for updating the disk. Once the cache is warmed up performance should be just fine.
Re:Optane Memory – A bridge between DRAM and storage
by wickedlogic on Monday March 27, 2017 @01:26PM
(#13969789)
Reaction to 2nd video: It is like a kitchen, and a panty, with a sous-chef which is like a conveyor belt. I mean if this runs ruby and also uses knife, how fast could this really be. ;)
Re:Optane Memory – A bridge between DRAM and storage
by wtallis on Monday March 27, 2017 @08:56PM
(#13973097)
If you've got suggestions for how to meaningfully and realistically compare caching of a Windows boot volume against a Linux cache solution, please share. My usual suite of tests won't work well for this, and I'm open to suggestions for what to include in my new test suite.
Re:Optane Memory – A bridge between DRAM and storage
by wallacoloo on Monday March 27, 2017 @03:33PM
(#13970919)
It does. Somebody else linked to the Semiaccurate article higher in the comments where they discuss this. It looks as though the per-bit endurance of the Optane drive is a bit less than modern flash, but they counteract this by having the drive's firmware reserve some blocks that it can remap over failing blocks.
Re:Optane Memory – A bridge between DRAM and storage
by solotronics on Monday March 27, 2017 @02:01PM
(#13970131)
my understanding was one goal of this would be replacing DRAM with 3D XPOINT so theoretically your computer would have 1 TB of this new memory in its RAM slots.
Re:Optane Memory – A bridge between DRAM and storage
by valarauca1 on Monday March 27, 2017 @02:09PM
(#13970188)
Your understanding if a bit flawed, and largely based on marketting.
Intel has hinted XPOINT will be usable in DRAM slots, but has yet to announce a product.
Intel has stated XPOINT will replace DRAM buuut based on it's marketing material it won't. Having latencies go from 14nano-seconds to 9000nano-seconds isn't acceptable for a DRAM replacement.
Re:Optane Memory – A bridge between DRAM and storage
by dsr_ on Monday March 27, 2017 @05:15PM
(#13971838)
Badly. Suppose you have a 16GB laptop and the 16GB optane. When you close your laptop to take it to a mid-morning meeting, you've just done a whole drive write. Do it again at lunch, twice in the afternoon and again when you get home, you're at 5 of your 6 drive writes.
Now imagine you open your laptop 8 or 12 times a day...
http://www.anandtech.com/show/11227/intel-launches-optane-me...
> The Optane Memory products are squeezing into a relatively small niche for limited budgets that require a lot of storage and want the benefit of solid state performance without paying the full price of a boot SSD. Intel notes that Optane Memory caching can be used in front of hybrid drives and SATA SSDs, but the performance benefit will be smaller and these configurations are not expected to be common or cost effective.
It doesn't sound that fast. Looks to be "slower" (throughput-wise) then modern NVMe drives (i.e. Evo 960 and obviously smaller) and clearly trying to bridge a price gap. Is the upside that it can act as an additional cache level without OS support? I remember the first time I used Intel Rapid Storage Tech (aka RST) ... it was also the last time as it's was convoluted with minimal (or no?) upside on Linux.
As an "enthusiast" I'm assuming that my Linux desktop with plenty of RAM + NVMe backed lvmcache will perform better.
I also assume that my laptop with reasonable RAM (because DDR (self)refresh sucks enough power to annoy me) + high performance NVMe drive also won't benefit.
Is this just marketing fluff?
> Keep in mind that consumer NVMe SSDs that boast throughput of 2GB/s or more generally do not reach their peak at low queue depth. Optane is supposed to be able to drive 1200MB/s read throughput at low queue depth (not sure why they listed QD4), so there is potential for some performance improvement here. Most consumer workloads never get out of low queue depth territory, so this could have some small real world benefit. Write throughput, however, is critically low.
> More importantly, these Optane drive are gear more towards lowering latency than transferring large files. Where HDDs access the data on the order of 10s of mS and SSDs access data on the order of 1mS (give or take), Optane should be able to access data on the order of 1s - 10s of uS. Where Optane will be useful is high numbers of small file accesses (DLLs, library files, etc.).
> That all said, I'd just as soon leave all the extra complications, compatibility issues, and inconsistencies on the table and get that 2 GB/s sdd that you mentioned until Intel figures out how to make these more compatible and easier to use without requiring a "golden setup". I don't want to buy a new W10, Kaby Lake, 200 series based system just to use one of these. My current W7/W10/Ubuntu, Skylake, 100 series system should work just fine for a good while yet.
[0] http://www.anandtech.com/comments/11227/intel-launches-optan...
Reads are as low as 20micro-seconds [1] This is using an Samsung 860pro, so a generation behind the current generation.
[1] https://www.cs.utah.edu/~manua/pubs/systor15.pdf
Seems like Optane is a solution looking for a problem to fix. Normal M2 SATA's do fine: # dd if=/dev/sdb of=/dev/null bs=131072 1953586+1 records in 1953586+1 records out 256060514304 bytes (256 GB, 238 GiB) copied, 159.18 s, 1.6 GB/s
(cold cache on a machine with 16GB ram).
Can't find a better article right now than this SEmiaccurate rant [1] which is quite inflammatory, but does not push falsehoods AFAICT. But Intel has essentially walked back from the 100-1000x improvements it initially claimed to something that's barely better than NVMe in terms of performance and robustness seems far from initial claims too.
[1] https://semiaccurate.com/2017/03/10/intel-mislead-press-xpoi...
The first gen SLC - from the likes of STEC were as fast if not faster than anything we have now, they were just ridiculously overpriced.
[1] http://techreport.com/review/17269/intel-second-generation-x...
And is it legal ?
People have short memory (even tech journalists), especially when there is not enough competition.
If the gains aren't that significant, why would people buy it and why would PC maker want to use it?
---------
The Optane memory seems to have a niche in replacing DRAM servers. So instead of running Redis, MySQL, or Postgres on DRAM, you can run it on Optane instead.
Optane is cheaper than DDR4 but almost as fast (latency wise). Latency is a big deal when it comes to database and/or key-value pair workloads.
The price difference between 4 GB and 8 GB of RAM cannot be enough to compensate for the Optane module, but if the Optane combination uses less battery than the bigger RAM and if enough benchmarks and/or magazine performance comparison use cases coincide with tasks where the hybrid shines, then some low end manufacturers might successfully sell the RAM downgrade as an upgrade option.
With idle power specified as 0.9-1.2 W, I'm not expecting the Optane-based solutions to come out ahead on any battery life tests. PCIe power management is a horrific mess.
Also if you are already mmap'ing some big file which needs frequent access, I do not see how this will help ?
An interesting interesting system design might be a 2GB main memory with a 32GB swap partition on this device. That would be interesting because in idle mode it uses less power than an equivalent amount of DDR memory. So you could potentially build a laptop with good performance on larger memory tasks but longer standby battery life. It would also be interesting paired with an equivalent amount of DRAM where it mirrors main memory so you could have 'instant' off and 'instant' on type booting and shutdown.
At larger pool sizes this becomes interesting as a non-volatile index for NoSQL and other databases. Intel's claim has been in the past that its density can exceed SSDs by a healthy margin.Not seeing that in these products but time will tell.
I've been watching them struggle to productize this tech for about 2 years now[1]. Now to see if they can get it to live up to the hype.
[1] https://arstechnica.com/gadgets/2015/07/intel-and-micron-unv...
Seems doubtful they would reach that goal until SSDs hit some kind of hard plateau (kind of like HDDs did).
Toshiba plans to make a 100TB SSD over the next couple of years, and I believe Samsung has already announced a 16TB SSD. I doubt 3D Xpoint can catch-up with those in the given time frame.
3D NAND turned the clock back on horizontal scaling and cell sizes by several years, and now the only scaling opportunity is to increase layer count. That's not going so well. Samsung started with 24 layers but didn't ship retail products until they hit 32 layers. Their 48 layers was about a year late and caused several products to be scrapped entirely. Toshiba's on their third generation 3D NAND and it's only just starting to show up in the market. SK Hynix is doing worse than Toshiba. Intel and Micron started off reasonably well with their 32-layer and are working on ramping up 64-layer.
By the end of this year, all four manufacturers should be shipping 64-layer (72 for SK Hynix) 3D NAND in volume, or else it's time to admit that 3D NAND just doesn't scale up as well as we hoped it would.
There's some reason to believe 3D XPoint memory might be able to avoid the worst vertical scaling problems that are holding back 3D NAND flash. Each pair of 3D XPoint layers can be pretty much isolated from the rest of the layers except at the edges, whereas 3D NAND is building columns of bits that run through all 64+ layers and have aspect ratios that are quite high.
[1] https://en.wikipedia.org/wiki/Resistive_random-access_memory [2] http://www.4dsmemory.com
You lose SSD transfer speed, while gain _some_ hardware response latency.
NVMe response latency (on hardware side, not OS) is around 14-15micro-seconds on reads, and tops out at 200micro-seconds on writes [1]. Which is nearly 2-3 orders of magnitude below spinning rust. While Optane can halve this to 7micro-seconds on reads. The price/pref vs established NVMe tech is non-existent.
When I can spend $X/GB to get a 2000% performance bump, its hard to justify spending $X*4/GB to get 2050% performance bump. Most applications/data centers have yet to transfer to NVMe. Offering marginal gains a sizable markup is a risky strategy.
Furthermore Optane is nearly 2 orders of magnitude lower than RAM in latency, and 5 orders of magnitude lower transfer. So the idea of "non-volatile RAM" has yet to arrive.
---
Also "Transparent to the OS" should strike fear in your heart. The OS's job is to manage hardware, memory, and caches. Why drop this?
[1] https://www.cs.utah.edu/~manua/pubs/systor15.pdf
90us of raw device latency, then software overhead of 22-25us (2-3us driver, 6-7us kernel, 14-15us user space).
So, the 6us access latency quoted in the Anandtech article would be significantly faster than NVMe SSD.
Too small to provide much of an cache, when you can have 256GB of SSD for the same price.
It is a pity that the marketing docs don't address how this impacts on write consistency. Instant write sync is the primary advantage of hardware RAID, but still, how much better is this than a normal SSD?
These first generation products don't actually have better endurance, though.
"Q4: Will Intel Optane memory also accelerate an SSD? If so how much?
A: Yes. Intel Optane memory can be used to accelerate any type of SATA-based storage media, including SATA SSDs, However, the performance benefit of adding Intel Optane memory will be greater on slower storage devices like an HDD versus a relatively faster storage device like a SATA-SSD. Additional performance data is forthcoming"
http://www.intel.com/content/www/us/en/architecture-and-tech...
Intel's Windows-only caching software for the Optane Memory product will refuse to work on any but the newest Intel platforms, but any other caching software will work fine on any platform that supports PCIe. If it turns out that small Optane caches are worth what Intel's charging for the hardware, then the existing third-party caching solutions will get tuned to make full use of Optane.
2. In the NVMe form factor, it supports older CPUs.
3. In the DIMM form factor, it will be far, far faster than NVMe, but intel isn't saying when those are coming.
The idea is M.2 speed > SATA, but the dynamic M.2 caching isn't native to the OS, so drivers are required to intermediate.
Pretty underwhelming relative to the hype.
https://en.wikipedia.org/wiki/M.2
But yes, nicer M.2s will perform better!
Maybe this makes sense if you need lots of space. Over 1TB SSDs are expensive.
I know the Optane SSD drives are fast -- is this just device-side driver software that allows you to virtually expand available "RAM" and mmap it to the SSD?
Why just i7 CPUs? Is there a hardware component?
"Just download and install the driver..." (Linux?)
On the enterprise side, Intel is offering the Memory Drive Technology software that is a minimal hypervisor to present a pool of memory equal to about DRAM+85% of Optane, with the actual allocation managed by the hypervisor and opaque to the guest OS. The hypervisor can run on several generations of Xeon processors.
I have tested on these devices, as a datastore for databases, and it has benefits. If you're writing a lot, and/or are very latency sensitive.
And about Linux; somewhere in the PDF with the technical details it states the requirement that your OS has to be 64 bit Windows ..
Optane products use 3D XPoint memory, which is not 3D NAND flash memory or any other kind of flash memory. It's probably a phase change memory, but Intel won't say so we have to wait for Chipworks to put it under an electron microscope.
The Optane Memory product for consumers is entirely centered around the use case of running Intel's SSD caching software for Windows, which requires a Skylake or newer chipset plus boot firmware that will only actually be available for Kaby Lake systems. The hardware feature they're relying on is a disgusting hack that Linux users want to stay far away from (and can, since there are generic caching solutions for Linux that work atop any block device).
I'll believe it when I see the spec and the code.
3D XPoint DIMMs are on the roadmap for next year.
How would a hypervisor sulution work? Trap and emulate all accesses?
This is already kind of doable with SSD, but if optane has an order of magnitude less access time it would of course make it even better for this use case.
LMDB, for example, would be a great fit with Optane already. Really, anything that uses mmap() is.
But for consumers? I am not sure if they can feel the difference between a decent SSD drive and Optane Memory. It fits the role of Page File. So you could always have 2x - 3x the Optane Memory as Page Cache. System with this would means less Memory required, and saves battery. While the controller for SSD could be less complicated and focus primarily on Seq/Read Write. SSD could also be DRAM-less, save cost and battery.
But even then, $44 for 16GB is just hard to convince most consumers.
Good random-write performance with 4k/8k blocks at QD1 is what matter for SSD's, not if they can write 2500MB/s to a volatile cache.
Recommended read http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test...
Be sure and check queue depth when comparing. Apparently their claim to fame is full speed even at low queue depth.
So take a CPU with more memory channels on it, run at least 1 Optane per channel, run hundreds of VMs on it, everything runs from this slower RAM, but it is in memory and you only touch the SSDs for updating the disk. Once the cache is warmed up performance should be just fine.
[1] http://www.intel.com/buy/us/en/audience/optane-memory
Intel has hinted XPOINT will be usable in DRAM slots, but has yet to announce a product.
Intel has stated XPOINT will replace DRAM buuut based on it's marketing material it won't. Having latencies go from 14nano-seconds to 9000nano-seconds isn't acceptable for a DRAM replacement.
Now imagine you open your laptop 8 or 12 times a day...
Which OS does that? I know macOS doesn't, but it still manages to recover documents in the event of a crash thanks to NSDocument autosaving anyway.