When SYSCALL added EFLAGS masking, SWAPGS was also created with the assumption that it would be always done when going between kernel and user mode and before enabling interrupts.