A manager with an attitude that engineers would pad their estimates to "enjoy the free time" is a guaranteed way to get good engineers to quit a team. Not only does it betray a complete lack of trust, but it means engineers are going to be pressured to lower their estimates, which never turns out well.