You can theoretically have a somewhat fast enough large JS app but it's going to be an uphill battle.

You have to make regular bundle analysis otherwise the cache won't work if you deploy too much and package updates and new additions are likely to break the performance analysis you've just made before.

Less JS = better performance is a simplified model but very accurate in practice in my opinion, especially on large teams.