I think it's because 3.0 and 3.1 were basically "we broke everything, it's slower, and there are no new compelling features." The versions after that have been fine, but I think the first two versions were such clunkers that it created a bad schism, and now that there's a schism nobody wants to support two languages, so people just stick with 2.7. I think if they had just put some new feature that people had to upgrade for in 3.0, they would have avoided this mess, and people would have grumbled for a few months and adapted.

I don't think the lesson is "never break compatibility". The lesson is "don't compete with yourself by releasing a product that is actively worse than your current version"