Per Collins: "If you describe someone or something as part of the furniture, you are suggesting that they have been somewhere such as their place of work for such a long time that it is hard to imagine that place without them."

It's neutral, really.