SKUs of Haswell with the GT3e graphics configuration include 128mb of on package DRAM, which is intended to give more memory bandwidth to the GPU, but it also acts as a cache for the CPU.

Based on what you're saying, it seems as though AVX-512 and future models with larger, faster embedded DRAM might play very nicely together.

I should have been clearer. I was talking about the cache hierarchy and CPU memory pipes. I suppose there's a divergence in main memory too, with CPUs using DDR3 and GPUs using GDDR5, but as you say you can just throw cache at that problem.

Larrabee itself is defunct, but one of its successors is the "Xeon Phi" line, and the Knights Landing generation of that will have AVX-512.

Memory technology keeps improving as well, and will have improved by late-2014/2015. Regardless, even if an eight-core chip would be starved for memory running eternal AVX-512 instructions, the advantage would be power and heat savings with a single core running a limited set of computations as quickly and efficiently as possible, doing the most with the least. As the power profile of chips gets smaller and smaller, that would be the biggest advantage.