The GCC docs seem basically useless. I have no idea if GCC plans to keep it like this forever but, right now, this is about x86's CET features. CET is an Intel spec that adds two very loosely related features: Indirect Branch Tracking (IBT) and Shadow Stacks (SHSTK). Neither is supported yet by upstream Linux.

IBT is a rather weak indirect-branch-and-call protection scheme. IMO it's weak enough that it has dubious value, and software-only schemes can be a lot stronger. The main benefits of IBT are that it can guarantee that a program won't, even when attacked, jump to the middle of an instruction and that, since IBT is a hardware mitigation, it can provide a degree of protection against Spectre-like attacks. Specifically, the CPU won't speculatively branch to an address that isn't an indirect branch target.

SHSTK is a very strong return address protection. With SHSTK on, it's really quite difficult to convince the CPU to let RET branch to anywhere other than were the original CALL came from. This even integrates with the page tables -- the memory it uses cannot be written by normal stores. The latter part is a large part of why this isn't upstream yet: the x86 pagetable format is already a technically-indebted mess, and SHSTK makes it worse in ways that are quite nasty. It'll get sorted out eventually.

You can find SHSTK in Zen 3 (yes, AMD brought Intel's feature to market before Intel -- go AMD!) and, apparently, in one Tiger Lake part. The latter also supports IBT.

You can play with the GCC support by compiling a trivial program like this:

    void test(void (*fn)())
    {
     fn();
    }

(no Godbolt link because I didn't figure out how to extract the right output from Godbolt.) If you compile with -S -fcf-protection=branch, the indirect jump is unchanged but the function itself gets an endbr64 (or endbr32) instruction at the beginning. That's a new special NOP that means "this is a valid indirect branch target". The CPU will only allow indirect branches that land on an endbr64 instruction, so the actual indirect jump in this function is unchanged. I think GCC is clever enough to omit the endbr64 if it can prove that the address of the function is not taken.

The other interesting thing that GCC does is to emit this:

            .section        .note.gnu.property,"a"
            .align 8
            .long    1f - 0f
            .long    4f - 1f
            .long    5
    0:
            .string  "GNU"
    1:
            .align 8
            .long    0xc0000002
            .long    3f - 2f
    2:
            .long    0x3
    3:
            .align 8

The 0x3 changes depending on the -fcf-protection option. This is a magic incantation for the benefit of the linker and kernel declaring whether this object supports IBT and/or SHSTK. This way you don't crash if you link a non-endbr-instrumented file into a IBT-using program.

Note that SHSTK support doesn't affect the code at all. The CALL and RET magic is handled by the CPU, and library functions like setjmp use special new instructions in order to keep working.

There is also CPU support for CET and IBT in the kernel. SHSTK in the kernel is a huge mess. Specifically, the x86 architecture is starting to fall apart -- there are various ill-conceived features like SYSCALL, NMI, the machine check architecture, and some fancy virtualization features that, collectively, conspire to make it extremely complicated to correctly handle all types of interrupts and interrupt-like events. SHSTK-in-kernel makes it worse, and I don't think anyone wants to try to deal with this yet. I and others have been badgering Intel and AMD for quite some time to do something about the mess that x86 kernel entries have become, and so far there are no results.

Thank you for taking the time to write this very in-depth comment, I can say I understand some of these words now!

The introduction of https://software.intel.com/sites/default/files/managed/4d/2a... has as much detail as I could ask for. Seems the GCC flag only enables the Intel implementation for now

Do you know what this flag considers to be a "valid" control flow transfer? The documentation makes it sound that this feature can safely be enabled all the time. But the linked article suggests otherwise...

It's safe almost always, unless A) you're doing something really undefined-behavior-ish and strange... or, here, for iPXE, I'm guessing the problem is B) you're running code that it emits assuming it's running on the Linux kernel in a fully initialized environment in a preboot/embedded environment.. and still expect setjmp/longjmp to work.

That is, it doesn't make sense with -ffreestanding, but Ubuntu enables it in that case.