Falling into the 3-4 crowd myself I also agree with this. Changing emails - perhaps in an automatic fashion - to use AMP is going to do nothing but harm to anyone outside Google. Image inclusion the AMP way [1] alone will ruin HTML email for other services. If Google really wanted to improve emails they would push for full compliance HTML5 support. Because if emails could be built with grid layouts, content could be more semantic, you wouldn't need a plain text version, we'd use less mark up and save on server space and bandwidth.

[1] https://ampbyexample.com/components/amp-img/