Take the natural logarithm of the comment karma, and then use as the weight in a weighted shuffle algorithm?

Higher-voted comments will then be more likely to move up, but not guaranteed.

(You might want to consider putting a thumb on the scale to distinguish between comments which are low voted because they're uninteresting, and comments which contain flame-bait words etc.)