The OS cares, sadly. When the OS, running one one CPU, instructs another CPU to start (which happens — your system boots into your OS and also resumes from suspend with only one CPU running OS code), it sends a message called SIPI to that CPU. The CPU responds by running OS code, in real mode, at an address <1MB. So the OS needs to find valid memory below 640kB that doesn’t conflict with any other horrible legacy use.

Also, the “LA57” 5-level page table design has a little whoopsie: you can’t switch between 4 and 5 level page tables without exiting paged mode entirely.