At its root, I think my objections to AMP boil down to a few things:

On a technical level:

1. It's buggy and weird on iOS.

2. I'm not convinced I care about a few seconds of loading time enough to justify the added complexity of making this kind of prefetching possible. Additionally, this seems like a stop-gap that will be rendered unnecessary by increasingly wide pipes for data.

On a philosophical level:

3. It gives Google way too much power over content.

4. I want the option to turn it off completely because of points [1] and [3], and because I fundamentally want to feel in control of my internet experience.

Edit: The point about SXG making AMP URLs less likely to get copy/pasted to other mediums is a key benefit I hadn't considered and will likely make avoiding AMP outside of Google search easier.