FYI, if you cancel Prime but want to keep that card, they will downgrade you to a blue Amazon branded card that still gets 3% back at Amazon. I don't know if you still get any discount at Whole Foods since I might only go there about once a year at most.

So if Prime is failing you in other ways, don't feel like you need to keep it to keep your discount.