It's not the object's mass that you care about, but its ballistic coefficient [1]. An object which is very heavy, but also very big, will deorbit just as quickly.

[1] https://en.wikipedia.org/wiki/Ballistic_coefficient