The point is that the leap seconds don't actually help astronomers because they need sub-second accuracy anyway, so they're going to need a more exact time and they won't really care whether it says "12:47:45.435" or "12:47:44.435". So it's a lot of work for dubious benefit.

Also, we "suffer" from leap seconds not because we did a bad job designing the system, but because the Earth's rotation is slowing due to tidal forces from the moon, so an earth day today is already a tiny fraction of a second longer than it was when we defined a day to be 24 hours. So we have a couple of choices: change the length of an hour (and minute, and second...) to match the Earth's changing rotational speed; let solar time and clock time gradually drift apart; or add leap seconds to keep them roughly in sync. The first option is clearly insane, since having our measure of time vary over time would be a huge inconvenience. Out of the 2nd and 3rd, options, we have chosen the 3rd. A lot of people are arguing that the 2nd is really a better option. In the (very very) long term, it almost certainly is. Once the Earth's day becomes 25 hours, we'd be adding a leap hour every day!

This is my understanding of the situation, anyway. Anyone with more experience here, feel free to correct me.