> If all your pointers are 64-bit aligned, all your variables are 64-bit aligned and your processor isn't any faster processing 16-bit numbers - if it even have instructions to process those - than 64-bit numbers

Memory bandwidth. If the processor can read a single 64-bit integer in a clock cycle, it can read 4 16-bit ones just as well. Memory is slower than the core.