Seems to me the answer lays in having the compiler more tightly bound with the nature of the OS - such that, for example, adjustable stack sizes and generalized metrics about application performance, can be used to make apps fill the cache-line most optimally.

In Embedded/Safety-Critical works, such metrics are available; often-times though, compiler/OS levels of support/integration/inter-operation are insufficient to high-performance tuning .. and of course, for a lot of industrial applications, not necessary (or encouraged, at least) in the first place.

However I wonder what an attempt such as the one to make an exclusively Lua-based OS (and, thus Application-Set) might produce in terms of optimizing cache line fills; I say "Lua" because it of course has a pretty good mechanism for determining memory usage patterns ("all elements", "each element", "diverse access patterns" .. something that tables and metatables can easily assist) .. and with a strong App base on a LuaOS tweaked specifically for the cache/CPU combination its running on, perhaps we will see some interesting new endeavours in this department ..