Besides obvious documentation, tools and dev.support costs, also compatibility cost. Currently Intel/AMD can change the microarchitecture however they see fit, potentially even between revisions, as long as the instruction set stay stable.
Changing the microarchitecture itself is not trivial either, and you can bet that they've got it all documented internally along with tools and whatever else. They could just open it up, as in post everything on their website for download --- they don't have to provide "support". The only real reason they don't is because there are things in there that they don't want the world to know.
Major changing is not trivial. However I can see how they can implement minor tweaks between revisions that don’t affect their main interface (i.e. AMD64 instruction set + extensions) but do affect microcode.
> you can bet that they've got it all documented internally
I’ve worked in a couple of large US software companies. I won’t point fingers but the internal documentation I’ve worked with wasn’t OK. The only exceptions is when the company provided a public API. When that was the case, they allocated resources (developer’s time, plus sometimes a technical writer position) and did the job.
> along with tools and whatever else
Legal issues are likely. E.g. GPL-licensed code is fine in an internal tool, but if you want to distribute it, you have to also distribute the source code of everything else that’s linked. Sometimes you don’t want to, other times you just can’t (if it links to a commercial library you’ve bought).
> because there are things in there that they don't want the world to know
I don’t eliminate the possibility but I don’t think that’s likely. As you see, there’re pragmatic reasons (i.e. money) why they aren’t doing that.
The good thing is, if anyone (Intel, AMD, I dunno, Apple, Qualcomm) will do that, and it will indeed deliver a great value to the users (such as free ASAN), the rest of them will do the same really fast, because competition.
The development cycle could be littered with dead processors, but worse for Intel, if a widely used microcode has a very infrequent chip stressing flaw it will manifest as “Intel chips aren’t reliable.”
EDIT: http://vswww.kaist.ac.kr/ENGLISH/old_lecture/ee878-2002/lect...
It probably does not imply that the chip is indestructible through microcode though.
Selecting a register through a tri-state based bus demands a circuit with a O(1) critical path. Selecting a register through a logical selector demands a circuit with O(log n) critical path, and a large constant factor.
Exactly this is why CPU vendors won't do anything in that regard. They don't want people to have a way to avoid upgrading the hardware by buying a new processor.
You can implement entirely new instructions in microcode.
and how much are you really going to enable? some funny bit ops? some explicitly interleaved fetch and computation? its pretty cool to try to understand things, but is it worth it for Intel? alot of stuff seems to baked in hardware too...pretty sure the cache coherency protocol isn't on the table...maybe some different concurrency primitives?
They are default the now though. As far as I know hardwired control logic wasn't used for very long before people moved to using microcode. Maybe I am misunderstanding that part of your comment?
Can we assume you must be kidding too when you use the phrase "Normal CPU architectures"?
Soft flashed roms must be cryptographically signed anyway... the main danger is surely in making various types of timing attacks or side-channel attacks substantially easier to derive, usually those attacks are inferred with great effort. The argument of security through obscurity is more complex compared to higher level software, it's not purely about finding bugs and refining implementation.
As the speakers in this talk point out, open microcode would allow extending the CPU with brand-new capabilities, including not only high-performance instrumentation frameworks, but also, I think, nearly-free ASAN-like and CFG-like runtime safety checks.
Is security a reason not to open the microcode architecture? I don't think so. I know this talk happened at CCC, but I don't see the security consequences of open microcode being severe: if you're in a position to install a microcode update, you've already won.*
Backward compatibility? Sure. Microcode can change generation-to-generation, or even stepping-to-stepping. Public microcode programs would have to be written with the understanding that no stability is guaranteed. But you can still get a lot of useful work done that way --- for example, people write Linux kernel drivers all the time.
Trade secrets? Eh. Does it really matter whether a processor has 32 or 64 internal temporary registers? I'm not sure what secrets proprietary microcode might protect against an adversary that has a dedicated chip design team and electron microscopes.
So, yeah, this talk represents impressive work, but none of this work should be necessary. Then again, if wishes were horses, beggars would ride. We can't even get Intel to ship a processor without the damn management engine.
* Granted, microcode backdoors have the virtue of being subtle and easy to hide. This paragraph doesn't apply if microcode updates let you break the processor's internal security model, but it sounds like, from the talk, that microcode obeys security invariants.