The little errors do compound, but the errors have been made progressively littler; a modern ring-laser gyro INS has a drift of one millidegree per hour or less.

Or you can add an external correcting factor, such the Trident's astronav system that takes star-shots to recalibrate the INS.