That is basically the definition of hardness. It doesn't make gems indestructible; they can be worn down like anything else. We cut diamonds even though we don't have something harder to cut them with; it's just much harder than cutting softer stones with diamonds.

A cut gem's sharp corners will be worn down over time by the paper the gem is kept in. Paper's not that hard.

(And of course, even if scratching were completely impossible, gems are still easy to smash. There are all kinds of deformations that aren't scratching.)