A manager with an attitude that engineers would pad their estimates to "enjoy the free time" is a guaranteed way to get good engineers to quit a team. Not only does it betray a complete lack of trust, but it means engineers are going to be pressured to lower their estimates, which never turns out well.

Absolutely not. It's hard enough for a seasoned senior engineer to estimate something, let alone a non-technical manager. (Of course, said seasoned engineer might be in a managerial role now but it's still an engineering task.)

Only the person who does the job can do the estimate. Estimation by third-party is no better than rolling a dice.

Usually it's a team effort and not just one person. Yes the manager should have say.