The point is that if I'm using stuct literals, I want the compiler to stop me for those structs.

I'm explicitly rejecting the idea that all struct changes should be possible without producing compiler errors. Compilers errors when the guarantees your code is based on changes is a feature, not a bug.