Is that a fundamental limit, or just how they’re currently built because they don’t need to do anything better?

I don’t know a whole lot about spy satellites, but the existence of things like spy planes and accurate ICBMs suggests the latter.

For small sats in low orbit it can be a very big deal. Before you can start snapping pictures accurately you need to confirm the actual orbit. You let the thing whip by a couple times to confirm its speed and altitude. Then it will likely be spinning a little. You need to wait for the aerodynamic forces to smooth that out.