I think a single E core can handle all the background processes when they're not doing any real work. I currently have around 250 processes and 3500 threads running on my computer, and the CPU utilization is 1%. It would probably make sense to use an extremely power efficient core that's only used for background services, one core that could be clocked much higher than all the rest for the foreground thread(s), and maybe some combination of E and P cores for the rest of the threads doing actual calculations. But adding lots of simple E cores for parallel work feels a bit like Larrabee.

It's clear that we need something in between regular quad core CPUs, and GPUs connected through PCIe. Some kind of parallel compute unit should be standard in pretty much all CPUs (or SoCs) eventually.