Hmm, this isn't really so accurate, there's a reason racing yachts try to reduce weight in every possible way they can, it does make a difference. You could say that adding a carbon fibre sail also makes no difference to weight versus the keel of a boat like Sayonara or Brindabella etc, but it's still worth saving the weight for the best performance.

And also, your point about top speed isn't wholly accurate either, you can get 36 foot boats now which can make 15-20 knots, which used to be thought to be almost impossible. But this is possible now with the advances in boat building techniques and materials.

Waterline length obviously makes a difference but if your boat is heavier, the wetted length and drag are going to be much greater. Also, lighter boats can crest over heavy seas in a way heavier boats can't reducing the slowing effect of a heavy sea, so even out at sea without making any maneuvers, you gain performance. Also, if you are tacking then acceleration is kind of a big deal, since you lose speed with every tack and have to regain it.

You should look at some of the maxi yachts and see the speed and acceleration they are capable of and understand that weight really does make a difference.