They made a big deal out of it in the movie, that the stones had to be returned to the exact time they were taken from, or else they would be changing the future of the old "head".