On-package memory is not faster. I suspect it is more power efficient though.

It makes it easier to get to a particular clock speed. The geometry, interconnect lengths etc are all tightly controlled, the noise is less because you're not on the main PCB and you have interconnect options that aren't whatever your PCB process is (e.g. commonly gold wires).