ECC everything?

Yes. And most server hardware is already at least ECC ram. You may still want some light radiation shielding to prevent the worst, maybe some heavier shielding for solar flares. But beyond that, simple error correction can be baked into the software - ecc the bootloader and filesystem and you are mostly good to go

Isn't ECC only applied to RAM? Do any current gen CPUs have ECC cache?

I can't comment on AMD or Intel but Apple Silicon definitely uses ECC for at least the system level cache. On top of that it performs cache healing (swapping out bad lines for spares) on every cache level every time the system boots.

Geostationary orbiters operate there during the day but this concept would position systems 100x closer to earth and well inside the protective envelope.

In LEO, there is a lot of testing and mitigation you can do with your design to help reduce the chance and impact of radiation single events. For example, redundancy for key components, ECC for RAM, supervisor hardware, RAID or other storage tooling, etc.