This is one of the big points Ben Graham makes in the classic book The Intelligent Investor -- https://en.wikipedia.org/wiki/The_Intelligent_Investor

Essentially -- unrealized losses don't matter unless they are uncover-able losses.