The problem is when you tell your manager it'll take you X number of weeks to finish a task, and your manager trusts that you're working hard and that the task simply requires X weeks of work... but in reality you're barely spending any time on it every day, and could easily have done it in a few days. Meanwhile, the rest of the project is moving slowly because they need your piece to complete.

If, on the other hand, the manager's trust has no relevance here, because you are not setting your own time-work estimates, then have at it!

This is kind of a wild take. Estimates should be reviewed like anything else on a team, and if the estimate doesn’t fit the timeline, the team should try to either split up / parallelize the work, drop scope, or look for a creative solution (usually involving taking on some strategic tech debt).

Work planning should be a team effort.

This is completely consistent with what I said. Even as a team effort, you want your manager to trust when you say something has some non-obvious complexity. And the manager wants to trust their team members.

But my comment stems from someone above who said they do the minimum amount of work possible. This sucks for the team, who is working hard and trying to make progress, meanwhile this person is barely doing any work every day in between gaming or reddit or whatever, but (presumably) still joins stand-ups and Slack to chime in with whatever "complication" he's having to work through that day, and always gives the impression that their work is super tricky.

You might be misunderstanding what "minimum amount of work" means. The OP might simply be referring the minimum amount of work achievable without overexerting oneself via extra hours, working weekends, under stress, etc. Someone's minimum amount of work can be hard work. They literally said "I will always finish my work in the time it is expected to be finished." So if that amount of time is not satisfactory, it is up to the manager to ask for the time to be shortened- and then that becomes the new minimum.

Would you use the same argument if your company could easily pay you 2x your current salary but just pays you x?

It’s also a problem when the manager can’t tell if an estimate is bogus or not. It could be exaggerated. It could also be overly optimistic. To be a decent dev manager you need a feel for whether it’s in the right ball park. Or know when to bring in someone else who does.

IMHO (having managed small teams) a manager of technical team members needs to be able to accurately estimate how long something will take and the variance on that estimate. This is a core part of the manager's job. If a manager isn't technical enough to know how to do the tasks themselves if they had to, that manager should not have a job IMHO. This is part of what sets engineer-led companies like SpaceX apart from their "dinosaur" competitors and why SpaceX seems to be 10x better in output.