I had a friend who was very healthy and fit but, like you, experienced elevated heart rate (and some loss of energy) for a few days after hard exercise days. Turns out, he had a partial blockage in a cardiac blood vessel. It was not blocked enough to be noticeable under normal workloads. But under hard work loads, part of his heart did not get enough oxygen and experienced damage. Eventually he experienced a heart attack, which is when it was caught and stented. Every other vessel was clear... it was just one of those flukes.

Who knows if you have the same issue, but I would suggest it's worth getting checked out.