Caffeine is a vasoconstrictor. The headaches are from your brain trying to adjust to the blood vessel change. Quitting even a mild habit can shave some points off your blood pressure.
If you drink soda or add cream/sugar to your coffee or tea, you may also find yourself losing some weight from stripping out the empty calories.
That said, my consumption levels have been sane for a few years now. I understand the negatives of over consumption but I don't fully understand the case for elimination?