I don't think this really answers your question since I never had the symptoms you or the OP did, but I did have caffeine-antagonized panic attacks that (mostly) subsided after I quit caffeine entirely. I think I went something like 13 years without having any substantial amount of caffeine outside whatever was present in decaf tea or coffee (and often avoiding coffee entirely).

Recently, within the last 10 months or so, I've gone back to drinking it without much effect. The only significant difference in my case was that I started a regular and fairly intense exercise regimen after my exgf dumped me. The way she did it was horrible, it was stressful, and made me lose a bunch of weight (for me, ~10 pounds) in less than a week, but the end result was that I recognized the benefits and decided to work to lose more and keep it off. I wasn't hugely over my ideal weight (25-30 over) but losing most of it helped with a bunch of different things, including this.

I don't know if this changed my ability to metabolize caffeine somewhat faster (I think it did; or minimized the effects), but before this I could feel my heart starting to act a bit funny within minutes of drinking anything with caffeine to the extent I'd wind up with a full blown panic attack. Now, I can drink it without noticing much other than the jitters (normal for me). Doesn't affect my heart rate unless I drink too much.

That said, I very slowly ramped up my caffeine intake over a few weeks starting with regular black teas. Maybe try switching up your exercise routine and slowly reintroduce it? The first couple cups I had over a week definitely upset my stomach slightly but it got better. Though, I noticed even decaf would do that as well. Doesn't now for whatever reason.

Did you normally drink it by itself or with something to eat?