Hardware makers make this hard for themselves. Having expensive devkits and requiring NDA's for doing the most trivial things creates a dynamic where the only players able to jump through the monetary and bureaucratic hoops are large vendors. Those vendors are unable to deliver true innovation because they have so much vested in the status quo.

> The problem is that most users (home & enterprise) just want things to work, they don't really care much how to get there and to have the best efficiency.

When it comes to research, no one cares that much about what home users and enterprise-users-small-enough-not-to-use-custom-software-stacks want now. Case in point: I don't think many IT mangers were that eager to switch to using ZFS in production when it was announced back in 2004 (and ZFS wad been under development for years at that time).

I've considered doing a PhD project pushing and stretching the boundaries of SSD firmware/operating system/filesystems because I think there's a lot of improvement that can be done in this area. The cost of OpenSSD that the sibling comments mention wasn't even that much of a problem. I seriously don't think someone not associated with a research department somewhere would have the time and/or know-how to do original research and implement a working prototype. Hell, I might get a devkit, but I doubt I'll do anything interesting and original at the same time. Which brings us to the actual problem:

Documentation and NDAs. For lots of ICs, microcontrollers, processors you can freely get hardware documentation, programming manuals, etc. For flash controllers and high-density NANDs? Almost nothing at all. Maybe some stuff can be reverse engineered and you get some documentation for OpenSSD. But the NAND manufacturer won't tell you stuff that's really, really important about things like failure patterns, which would allow you to optimize error correction and wear leveling for example.

Documentation is an issue indeed.

I don't think that you really need the inner information about NAND to do the original and innovative research. It really depends on the area you want to work on, the SSD firmware level might require that but the SSD makers are already on that route (some better than others). The other level is to not pay too much attention to the differences between NAND chips and just implement something at a higher level to push the hardware-agnostic smarts to the OS.

The OpenSSD also lacks documentation, last time I looked at it there was no info on how to do NCQ on the SATA interface, without which there is no talking about a speedy SSD.

Case in point for the cost: I looked at buying an OpenSSD (referenced in the article) to write my own firmware to a lowly 64GB device. Was quoted $2000. There is nothing else getting close to this openness on the market.

... and the prognosis for OpenSSD is terminal.

It seems to be geared only towards research insititutes and a specific one at that, most of the work using OpenSSD is concentrated on that one university.

It may happen in the server world. Engineering wins the day a whole lot more often than in the consumer market. ZFS et al are a good case in point.