It doesn't have to be that way. We use it mainly as a tool to track issues and move these issues through a workflow. When I was scrum master one of the first things I did was to stop tracking time and we didn't enter story points or anything. Personally I would be very interested in numbers how much things take to learn estimating better but I know the numbers will be abused by some managers so it's better not give them any data that could be abused.