You have a very small number of values that can stay in L2 cache on a 2 MiB boundary, 12 on Skylake. The data TLB is only 4 way. Modern high performance software is about cache management, and spreading your data across swaths of page-aligned memory without strong control of when you get scheduled out underutilizes that most precious resource on the CPU.