Publishers want revenue, not fast sites. I think that is the root of the problem.

Instead of AMP a better solution could be to add buttons to browser to disable JS/CSS/Web Fonts (the most useless innovation)/Cookies and let the user decide what they want to see.