Can someone explain the disclosure procedures that were actually followed? It seems like everybody freaked out at the same time. AWS had apparently already protected all but a "small percentage" of their VMs. If there was some kind of responsible disclosure taking place, allowing vendors time to upgrade, it seems like it wouldn't have had so much impact.

I think the disclosure procedures were abandoned because people outside of the loop had figured it out from Linux kernel upgrades.

Also by AMD's e-mail that all-but-disclosed the mechanism of Meltdown.

AIUI disclosure wasn't supposed to happen until tomorrow. Google broke the embargo because, as you said, people were starting to figure it out.

Breaking the embargo at 5pm EST was also coordinated with everyone who had been disclosed early. It wasn't anarchy.

How does a microcode update work? My sparse understanding is that instructions are physically encoded in the chips. That you could take a microscope and look at the different instructions.

Microcode is essentially a program that runs on the CPU below the level of the normal instruction set (machine code). It's firmware, and stored in some memory on the die (or in the EFI/BIOS, or loaded at or after boot time, depending on the system.) It allows for mapping a large set of complex instructions to short programs of fundamental microinstructions. A CPU without a hardware multiplier might still have an assembly (and machine code) multiply instruction, and simply implement a shift-and-add multiplication algorithm in the microcode.

I suggest Tanenbaum and Austin's "Structured Computer Organization" for a pretty good book about this (and other aspects of CPU design).

Back when people were mostly programming in assembly it was convenient for one instruction to do a lot of stuff, but it hardwiring a large instruction set poses design difficulties. So CISC chips have some special microcode memory that runs a program that converts the larger instruction set into the actual hardware instruction set. Intel's CPUs since the Pentium Pro allow writing to the microcode memory (with severe restrictions).

According to Wikipedia [1], CPUs have had microcode since the 1960s! However, this is mostly an artifact of CISC architectures. Since most programmers these mass-produce their instructions with a compiler rather than with artisinal, hand-crafted assembly code, CISC fell out of favor compared to RISC. Since the RISC instruction sets are simple (by design), there isn't much need for microcode. ARM chips typically don't have microcode (although they do break things into smaller instructions) [2]. But since Intel has to keep compatibility with 40 years of CISC instructions, they need microcode.

[1] https://en.wikipedia.org/wiki/Microcode

[2] https://superuser.com/questions/934752/do-arm-processors-lik...

Yes, except that processors are now incredibly complex beasts that have all sorts of optimizations and other features (hence....a lurking, unobvious bug).

So there is this layer of software that operates with this x86 interface on a hardware implementation of different instructions.

That was a very long time ago.

Microcode can be loaded by the BIOS.

Most of the microcode nowadays is loaded by the OS (not BIOS)

Today I learned something.

Doesn't that open a huge attack surface? This code would run super-privileged isn't it?

CPUs will only accept uploads of microcode if they can verify the digital signature, of which the private keys are only known to the CPU manufacturer (i.e. Intel or AMD). Probably it will also only allow upgrades, not downgrades, on a running system. So the only "attack" you can perform if you've gained root is to patch CPU security vulnerabilities by upgrading to a newer valid microcode, and then also only until the next reboot.

Can also be loaded at boot time by the kernel, etc.

Or even after boot time, I think -- you shut down one CPU at a time to load it.

See also discussion and hyperlinks at https://news.ycombinator.com/item?id=16086519 .