News for hackers. Stuff that matters.
Re:Compiler bug? Linker bug? Windows Kernel bug by pjc50 on Monday February 26, 2018 @08:24AM (#16465053)
I found my own one: Windows "structured exception handling" relies on code in the kernel that unwinds call stacks. This makes a whole bunch of assumptions about the layout of the code as well. On CE/ARM, we found that Visual Studio was sometimes generating code that used "bl [r3]" constructs which could not be unwound. The workaround was to use a different version of cl.exe that came with the CE dev kit and not the one that came with Visual Studio.

Oh, and if you have more than about 512 bytes of local variables in a stack frame and try to use CE remote debugging, that stops working as well.

Re:Compiler bug? Linker bug? Windows Kernel bug by slededit on Monday February 26, 2018 @01:37PM (#16467734)
Its the same on X64, although I think its better documented/understood on that platform.
Re:Compiler bug? Linker bug? Windows Kernel bug by trentnelson on Monday February 26, 2018 @03:57PM (#16468899)
Yeah that's one of the things Cutler played an important role in -- formalizing x64 prologue and epilogue constructs. The epilogue stuff is interesting, there's a small set of instructions that are allowed because the unwind code literally does opcode analysis to figure out if the current function is in its epilogue or not.
Re:Compiler bug? Linker bug? Windows Kernel bug by philiplu on Monday February 26, 2018 @06:03PM (#16469782)
Heh - that was partly me. The kernel team (i.e. mostly DaveC) mostly prescribed which instructions were allowed in prologues/epilogues. I was dev lead of VC's CRT team at the time (2000-ish), and got assigned to work on that and come up with an x64 version of the prologue/epilogue descriptors needed to unwind. I looked at what was done for Itanium, and simplified that waaaay down to what was needed for x64. I remember designing the ability to split prologues into multiple parts, to do something called shrink-wrapping, where you might not want to push all your saved registers all the time - there might be a subset of a function's code where a register needs to be pushed/popped. DaveC needed convincing that was a good idea.