Well, a compliant JPEGs will still open in any reader, but can look slightly different. In a low-resolution image, it could change some fine detail you really cared about.

Ages ago I did some bit flip tests on different image encodings for a school project. It was interesting to see how a single flipped bit would propagate across the image in encodings that relied on similarity to nearby pixels for compression. Really the effect was limited to the frame, but the effects could be striking.