most of the NVMe backplanes I've seen give full 'enterprise 2.5" drive clearance to the thing, so if they are actually as thick as consumer SSDs, as most current SATA 'enterprise SSD' are, there's plenty of room for a heatsink without expanding the slot. The supermicro chassis (and I've only explored the NVMe backplanes from supermicro) usually put a lot of effort into drawing air through the drives, so assuming you put in blanks and stuff, the airflow should be there, if the SSD are setup to take advantage of it.

You need to be more precise with your terminology. There is no such thing as a NVMe backplane. NVMe is a software protocol. The backplanes for 2.5" SSDs to which you refer would be PCIe backplanes using the SFF-8639 aka U.2 connector. None of the above is synonymous with the M.2 connector/form factor standard, which is what I was talking about.

edit: okay, I re-read what you said and yes, these won't support M.2 drives, if I understand what's going on here, and it's possible I still don't. (I have yet to buy any non-sata SSD, though I will soon be making experimental purchases.)

I was talking about these:

http://www.supermicro.com/products/nfo/NVMe.cfm

Note, though, it looks like if you are willing to pay for a U.2 connected drive, you can get 'em with the giant heatsinks you want:

http://www.pcper.com/news/Storage/Connector-Formerly-Known-S...

further edit:

http://www.intel.com/content/dam/www/public/us/en/documents/...

available; not super cheap, but I'm not sure you'd want the super cheap consumer grade stuff in a server, anyhow.

further edit:

but I object to the idea of putting SSDs on PCI-e cards for any but disposable "cloud" type servers (unless they are massively more reliable than any that I've seen, which I don't think is the case here.) just because with a U.2 connected hard drive in a U.2 backplane, I can swap a bad drive like I would swap a bad sata drive; an alert goes off and I head off to the co-lo as soon as convenient and I can swap the drive without disturbing users, whereas with a low-profile PCI-e card, I've pretty much gotta shut down the server, de-rack it, then make the swap, which causes downtime that must be scheduled, even if I have enough redundancy that there isn't any data loss.

Take a look at how much stricter the temperature and airflow requirements are for Intel's 2.5" U.2 drives compared to their add-in card counterparts. (And note that the U.2 drives are twice as thick as most SATA drives.)

M.2 has almost no place in the server market. U.2 does and will for the foreseeable future, but I'm not sure that it can serve the high-performance segment for long. It's not clear whether it will reach the limits on capacity, heat, or link speed first, but all of those limits are clearly much closer than for add-in cards.

>M.2 has almost no place in the server market. U.2 does and will for the foreseeable future, but I'm not sure that it can serve the high-performance segment for long. It's not clear whether it will reach the limits on capacity, heat, or link speed first, but all of those limits are clearly much closer than for add-in cards.

No argument on m.2 - it's a consumer grade technology. No doubt, someone in the "cloud" space will try it... I mean, if you rely on "ephemeral disk" - well, this is just "ephemeral disk" that goes funny sooner than spinning disk.

But the problem remains, If your servers aren't disposable, if your servers can't just go away at a moment's notice, the form factor of add-in cards is going to be a problem for you, unless the add-in cards are massively more reliable than I think they are. Taking down a whole server to replace a failed disk is a no-go on most non-cloud applications...

You're probably tired of hearing this from me, but if you distribute the storage, you can evacuate all the VMs off a host, down it, do whatever, bring it back up, and then unevacuate.

> You're probably tired of hearing this from me, but if you distribute the storage, you can evacuate all the VMs off a host, down it, do whatever, bring it back up, and then unevacuate.

Yes. The same conversation is happening right now in the living room, because security has forced three reboots in the last year, after having several years where simply by being xen and pv and using pvgrub (rather than loading a user kernel from the dom0) we haven't been vulnerable to the known privilege escalations. This is a lot more labor (and customer pain) when you can't move people.

No progress on that front yet, though.

And if uptime is that important, you can just buy a server that supports PCIe hotswap.

can you point me at a chassis designed for that?

OTOH if you can fit 24 U.2s but only 6 AICs in a server, maybe RAIDed U.2 will still win.

Only if you postulate that your drives won't be able to saturate their respective number of PCIe lanes. The total number of lanes in the system is fixed by the choice of CPU, and having drives with a total lane count in excess of that only helps when your drives can't fully use the lanes they're given or if you're trying to maximize capacity rather than performance.

2.5" NVMe is fine and more scalable than add-in cards (more slots) with increased serviceability. The equivalent add-in card is usually much more expensive than the 2.5" version.

I thought this was the point of SATA Expresses, but the last I read, SATA Express actually holds on par and even falls behind M.2.