What I'm wondering reading this is why song rights were so profitable in this time period. Why were McCartney and Jackson making so much money and why didn't the record labels keep the rights locked up tight if the rights were so lucrative long-term?

> record labels keep the rights locked up

They don't own them up front in the first place. No doubt some labels negotiate that as part of a deal, but they're separate domains to start with.

I don't think that's an answer but passing the buck. Why are they separate in the first place? Why aren't record labels, as part of standard modus operandi, been buying copyrights from bands? They are best placed to do this sort of thing, much better placed than random musicians who happen to get rich and then dabble in investing - record labels have (or had) the contacts up front, they had the money, the diverse portfolios to spread risk...

I think some of this stems from history. There was a whole music publishing industry far before 'record labels' existed (before recording existed, really).